Psychometric Evaluation of a Critical Thinking Skills Instrument on Renewable Energy Using the Rasch Model

Abstract

Critical thinking is an essential twenty-first-century skill that enables students to analyze information, evaluate evidence, and make reasoned decisions. Therefore, a high-quality instrument is needed to accurately measure students’ critical thinking skills. This study aimed to analyze the quality of a critical thinking skills instrument on renewable energy using the Rasch model. The study employed a descriptive quantitative design involving 75 tenth-grade students. The instrument consisted of 15 multiple-choice items developed based on Ennis’s critical thinking framework, including basic clarification, bases for a decision, inference, advanced clarification, and supposition and integration. Data were analyzed using the Rasch model through construct validity, reliability, item difficulty, Differential Item Functioning (DIF), and Wright Map analyses. The results showed that the instrument satisfied the unidimensionality requirement, with a raw variance explained by measures of 28.8%. Most items demonstrated acceptable fit to the Rasch model, although one item showed misfit and another required revision. The instrument achieved fair person reliability (0.67), fair item reliability (0.78), and very good internal consistency (Cronbach’s alpha = 0.79). The DIF analysis indicated that all items were free from significant gender bias. Furthermore, the Wright Map revealed a mistargeting condition in which the average student ability exceeded the average item difficulty. Overall, the instrument demonstrated acceptable measurement quality and can be used to assess students’ critical thinking skills on renewable energy topics, although several items require further refinement.