Evaluation of STEM Literacy Test Instrument on Elements, Compounds and Mixtures Using Multidimensional Rasch Analysis

Abstract

STEM literacy is crucial in the 21st century, encompassing science, technology, engineering, and mathematics, allowing individuals to understand and apply conceptual knowledge to real-life problems. Evaluation of STEM literacy instruments is essential for improving the quality of STEM learning. This study aims to evaluate the STEM literacy test instrument using multidimensional Rasch analysis. This study employed a quantitative approach, including a description to validate the research instrument. The instrument consists of 22 items measuring science, technology, engineering, and mathematics literacy, focusing on elements, compounds, and mixtures. Participants included 259 eighth-grade students randomly selected from public junior high schools in Balongpanggang and Benjeng, Gresik, East Java. Data analysis employed a Rasch model with ConQuest programs. Results indicated that the multidimensional model had better-fit parameters than the unidimensional model and demonstrated that the item's validity and instrument's reliability meet requirements. The interaction between students' abilities and items revealed that most students have moderate to low STEM literacy abilities. Additionally, the analysis found no differential item functioning based on gender differences. This research indicates that multidimensional Rasch analysis helps evaluate STEM literacy test instruments so that valid and reliable instruments can provide accurate measurements in improving students' STEM literacy skills.