Determination of Gender Differential Item Functioning in Tegal Students' Scientific Literacy Skills with Integrated Science (SLiSIS) Test Using Rasch Model
Abstract
The emergence of Differential Item Functioning (DIF) indicates an external bias in an item. This study aims to identify items at scientific literacy skills with integrated science (SLiSIS) test that experience DIF based on gender. Moreover, it is analyzed the emergence of DIF, especially related to the test construct measured, and concluded on how far the validity of the SLiSIS test from the construct validity of consequential type. The study was conducted with a quantitative approach by using a survey or non-experimental methods. The samples of this study were the responses of the SLiSIS test taken from 310 eleventh-grade high school students in the science program from SMA 2 and SMA 3 Tegal. The DIF analysis technique used Wald Test with the Rasch model. From the findings, eight items contained DIF in a 95 % level of trust. In 99 % level of trust, three items contained DIF, items 1, 6, and 38 or 7%. The DIF is caused by differences in test-takers ability following the measured construct, so it is not a test bias. Thus, the emergence of DIF on SLiSIS test items does not threaten the construct validity of the consequential type.
Keywords
Full Text:
PDFReferences
Aliakbari, M., & Sadeghdaghighi, A. (2011). Investigation of the relationship between gender, field of study, and critical thinking skill: the case of Iranian students. In Proceedings of The 16th Conference of Pan-Pacific Association of Applied Linguistics, Hongkong, 8th – 10th August 2011,
Ardianto, D., & Rubini, B. (2016). Comparison of students' scientific literacy in integrated science learning through model of guided discovery and problem based learning. Jurnal Pendidikan IPA Indonesia, 5(1), 31-37.
Balart, P., & Oosterveen, M. (2019). Females show more sustained performance during test-taking than males. Nature Communications, 10(1), 1-11.
Battauz, M. (2019). On Wald tests for differential item functioning detection. Statistical Methods & Applications, 28(1), 103-118.
Bond, T., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences ( 4th ed.). Routledge.
Cheema, J. R. (2019). Cross-country gender DIF in PISA science literacy items. European Journal of Developmental Psychology, 16(2), 152-166.
Chiang, P. M., & Tzou, H. I. (2018). The application of differential person functioning on the science literacy of Taiwan PISA 2015. Humanities & Social Sciences Reviews, 6(1), 08-13.
Choi, Y. J., Alexeev, N., & Cohen, A. S. (2015). Differential item functioning analysis using a mixture 3-parameter logistic model with a covariate on the TIMSS 2007 mathematics test. International Journal of Testing, 15(3), 239-253.
Coletta, V. P., Phillips, J. A., & Steinert, J. (2012) FCI normalized gain, scientific reasoning ability, thinking in physics, and gender effects. In AIP Conference Proceedings 1413, Los Angeles: 09 Januari 2012.
Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among five approaches. Sage publications.
Cuevas, M., & Cervantes, V. H. (2012). Differential item functioning detection with logistic regression. Mathématiques et sciences humaines. Mathematics and Social Sciences, (199), 45-59.
Demirtasli, Ç. (2015). A study on detecting of differential item functioning of PISA 2006 science literacy items in Turkish and American samples. Eurasian Journal of Educational Research, 58, 41-60.
Disenhaus, N. (2015). Boys, writing, and the literacy gender gap: what we know, what we think we know (Dissertation). Burlington: University of Vermont
Embretson, S. E., & Reise, S. P. (2013). Item response theory. Psychology Press.
Fakhriyah, F., Masfuah, S., & Mardapi, D. (2019). Developing scientific literacy-based teaching materials to improve students' computational thinking skills. Jurnal Pendidikan IPA Indonesia, 8(4), 482-491.
Fisher, A. (2011). Critical thinking: An introduction (2th ed.). Cambridge University Press.
French, B. F., Hand, B., Therrien, W. J., & Vazquez, J. A. V. (2012). Detection of sex differential item functioning in the Cornell critical thinking test. European Journal of Psychological Assessment, 28(3), 201-207.
Ganley, C. M., Vasilyeva, M., & Dulaney, A. (2014). Spatial ability mediates the gender Difference in middle school students' science performance. Child Development, 85(4), 1419-1432.
Gómez-Benito, J., Sireci, S., Padilla, J. L., Hidalgo, M. D., & Benítez, I. (2018). Differential item functioning: Beyond validity evidence based on internal structure. Psicothema, 30(1), 104-109.
Grunspan, D. Z., Eddy, S. L., Brownell, S. E., Wiggins, B. L., Crowe, A. J., & Goodreau, S. M. (2016). Males under-estimate academic performance of their female peers in undergraduate biology classrooms. PloS one, 11(2), 1-16.
Hanushek, E. A., & Woessmann, L. (2016). Knowledge capital, growth, and the East Asian miracle. Science, 351(6271), 344-345.
Harish, G. C. (2013). Critical thinking skills among ninth standard students in relation to gender, intelligence and study habits. International Journal of Education and Psychological Research (IJEPR), 2(3), 13-20.
He, J., & van de Vijver, F. (2012). Bias and equivalence in cross-cultural research. Online Readings in Psychology and Culture, 2(2), 2307-0919.
Hofer, S. I. (2015). Studying gender bias in physics grading: The role of teaching experience and country. International Journal of Science Education, 37(17), 2879-2905.
Hou, L., la Torre, J. D., & Nandakumar, R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the Wald test to investigate DIF in the DINA model. Journal of Educational Measurement, 51(1), 98-125.
Huang, X., Wilson, M., & Wang, L. (2016). Exploring plausible causes of differential item functioning in the PISA science assessment: language, curriculum or culture. Educational Psychology, 36(2), 378-390.
Hyde, J. S. (2014). Gender similarities and differences. Annual Review of Psychology, 65, 373-398.
Jong, C., Hodges, T. E., Royal, K. D., & Welder, R. M. (2015). Instruments to measure elementary preservice teachers' conceptions: An application of the Rasch rating scale model. Educational Research Quarterly, 39(1), 21-48.
Kendhammer, L., Holme, T., & Murphy, K. (2013). Identifying differential performance in general chemistry: Differential item functioning analysis of ACS general chemistry trial tests. Journal of Chemical Education, 90(7), 846-853.
Lisova, T. V., & Kovalchuk, Y. O. (2017) Gender Differences in Mathematics and Science Achievement of Eighth Grade Students in Ukraine on TIMSS 2011. European Educational Research Conference, Copenhagen: 21-25 August 2017 .
Lyons-Thomas, J., Sandilands, D., & Ercikan, K. (2014). Gender Differential Item Functioning in Mathematics in Four International Jurisdictions. Education & Science/Egitim ve Bilim, 39(172).20-32.
Mair, P., Hatzinger, R., Maier, M. J., Rusch, T., & Mair, M. P. (2019). Package ‘eRm’. Version 0.14-0.
Mari, L., Carbone, P. and Petri, D. (2012). Measurement fundamentals: A pragmatic view. IEEE Transactions on Instrumentation and Measurement, 61(8), 2107-2114.
McFarlane, D. A. (2013). Understanding the challenges of science education in the 21st century: New opportunities for scientific literacy. International Letters of Social and Humanistic Sciences, (04), 35-44.
McPeck, J. E. (2016). Critical thinking and education (1th ed.). Routledge.
Mesic, V. (2012). Identifying country-specific cultures of physics education: A differential item functioning approach. International Journal of Science Education, 34(16), 2483-2500.
Millsap, R. E. (2012). Statistical approaches to measurement invariance (1th ed.). Routledge.
Ministry of Education and Culture of the Republic of Indonesia.( 2018). Peraturan Menteri Pendidikan Dan Kebudayaan Republik Indonesia Nomor 37 Tahun 2018.
OECD .(2016). PISA 2015 Assessment and analytical framework: science, reading, mathematic and financial literacy, PISA, OECD Publishing, Paris. retrieved from http://dx.doi.org/10.1787/9789264255425-en
Ong, Y. M., Williams, J., & Lamprianou, I. (2015). Exploring crossing differential item functioning by gender in mathematics assessment. International Journal of Testing, 15(4), 337-355.
Preiss, D. D., Castillo, J. C., Flotts, P., & San Martín, E. (2013). Assessment of argumentative writing and critical thinking in higher education: Educational correlates and gender differences. Learning and Individual Differences, 28, 193-203.
Rachmatullah, A., & Ha, M. (2019). Examining high-school students' overconfidence bias in biology exam: a focus on the effects of country and gender. International Journal of Science Education, 41(5), 652-673.
Rahmawati, R. (2019). Efek sub-sampel pada deteksi differential item functioning (DIF) dengan metode regresi logistik. JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia), 1(1).21-30.
Ratini, R., Muchtar, H., Suparman, M. A., Tamuri, A. H., & Susanto, E. (2018). The influence of learning models and learning reliance on students' scientific lteracy. Jurnal Pendidikan IPA Indonesia, 7(4), 458-466.
Ravand, H., & Firoozi, T. (2016). Examining construct validity of the master's UEE using the Rasch model and the six aspects of the Messick's framework. International Journal of Language Testing, 6(1), 1-18.
Reilly, D. (2012). Gender, culture, and sex-typed cognitive abilities. PloS one, 7(7), 1-16.
Reilly, D., Neumann, D. L., & Andrews, G. (2015). Sex differences in mathematics and science achievement: A meta-analysis of National Assessment of Educational Progress assessments. Journal of Educational Psychology, 107(3), 645–662.
Roth, W. M., & Lee, S. (2016). Scientific literacy as collective praxis. Public Understanding of Science, 11(1), 33-56.
Rouquette, A., Hardouin, J. B., & Coste, J. (2016). Differential Item Functioning (DIF) and Subsequent Bias in Group Comparisons using a Composite Measurement Scale: a Simulation Study. J Appl Meas, 17, 312-334.
Rudolph, J. L., & Horibe, S. (2016). What do we mean by science education for civic engagement?. Journal of Research in Science Teaching, 53(6), 805-820.
Runnels, J. (2012). Using the Rasch model to validate a multiple choice English achievement test. International Journal of Language Studies, 6(4), 141-155.
Rusch, T., Lowry, P. B., Mair, P., & Treiblmaier, H. (2017). Breaking free from the limitations of classical test theory: Developing and measuring information systems scales using item response theory. Information & Management, 54(2), 189-203.
Rusilowati, A., Kurniawati, L., Nugroho, S. E., & Widiyatmoko, A. (2016). Developing an Instrument of Scientific Literacy Assessment on the Cycle Theme. International Journal of Environmental and Science Education, 11(12), 5718-5727.
Sabah, S., Hammouri, H., & Akour, M. (2013). Validation of a scale of attitudes toward science across countries using rasch model: Findings from TIMSS. Journal of Baltic Science Education, 12(5), 692-702.
Steinmayr, R., Bergold, S., Margraf-Stiksrud, J., & Freund, P. A. (2015). Gender differences on general knowledge tests: Are they due to Differential Item Functioning?. Intelligence, 50 (May–June 2015), 164-174.
Stoet, G., & Geary, D. C. (2012). Can stereotype threat explain the gender gap in mathematics performance and achievement?. Review of General psychology, 16(1), 93-102.
Strobl, C., Kopf, J., & Zeileis, A. (2015). Rasch trees: A new method for detecting differential item functioning in the Rasch model. Psychometrika, 80(2), 289-316.
Susongko, P. (2016). Validation of science achievement test with the rasch model. Jurnal Pendidikan IPA Indonesia, 5(2), 268-277.
Susongko, P., Widiatmo, H., Kusuma, M., & Afiani, Y. (2019). Development of integrated science-based science literacy skills instruments using the Rasch model. Unnes Science Education Journal, 8(3).
Tamassia, L., & Frans, R. (2014). Does integrated science education improve scientific literacy?. Journal of the European Teacher Education Network, 9, 131-141.
Taylor, C. S., & Lee, Y. (2012). Gender DIF in reading and mathematics tests with mixed item formats. Applied Measurement in Education, 25(3), 246-280.
Turiman, P., Omar, J., Daud, A. M., & Osman, K. (2012). Fostering the 21st century skills through scientific literacy and science process skills. Procedia-Social and Behavioral Sciences, 59, 110-116.
Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174.
Wang, C. C., Ho, H. C., Cheng, C. L., & Cheng, Y. Y. (2014). Application of the Rasch model to the measurement of creativity: The creative achievement questionnaire. Creativity Research Journal, 26(1), 62-71.
Wang, M. T., & Degol, J. L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge, implications for practice, policy, and future directions. Educational Psychology review, 29(1), 119-140.
Welner, K. G. (2013). Consequential validity and the transformation of tests from measurement tools to policy tools. Teachers College Record, 115(9), 1-6.
Wilson, K., Low, D., Verdon, M., & Verdon, A. (2016). Differences in gender performance on competitive physics selection tests. Physical Review Physics Education Research, 12(2), 1-16.
Woods, C. M., Cai, L., & Wang, M. (2013). The Langer-improved Wald test for DIF testing with multiple groups: Evaluation and comparison to two-group IRT. Educational and Psychological Measurement, 73(3), 532-547.
Yildirim, O. (2019). Detecting gender differences in PISA 2012 mathematics test with differential item functioning. International Education Studies, 12(8), 59-71.
Zwick, R. (2012). A review of ETS differential item functioning assessment procedures: Flagging rules, minimum sample size requirements, and criterion refinement. ETS Research Report Series, 2012(1), 1-30.
Refbacks
- There are currently no refbacks.
