Self-Reflection Assessment in Vertebrate Zoology (SRAVZ) Using Rasch Analysis
Abstract
Instruments that are valid, reliable, and have high consistency are needed to measure students’ self-reflection. The Self-Reflection Assessment in Vertebrate Zoology (SRAVZ) was developed to explore students’ self-reflection and abilities in the vertebrate zoology course. It is essential to test the instrument’s validity before measuring students’ abilities so that data bias does not occur. This study aims to determine the validity, whether the items are fit or misfit, and the difficulty level of SRAVZ items. SRAVZ is developed by ADDIE (Analysis, Design, Development, Implementation, Evaluation). The SRAVZ consists of 24 items tested on 135 students who have taken the vertebrate zoology course. Analysis of the Rasch model using Winstep version 4.5.2. The Rasch model shows the item reliability value at 0.97. The Cronbach alpha value at 0.94 with PTMEA Corr shows a positive value, unidimensional 48.1%. The separation index of 5.6 means that the level of grouping the items is very good. The mean square infit for SRAVZ was 0.59-1.96, and the mean square outfit value is 0.59-2.16. Data analysis shows that 24 SRAVZ items have 22 fit items and two misfit items (S3 and S5). Item numbers S3 and S5 must be excluded from the SRAVZ construction. Total items used to measure students’ self-reflection in the vertebrate zoology course were 22 items. The most difficult item is S3, and the easiest item is S6. Thus, the data indicate that the valid and reliable SRAVZ is in the good, effective, and high level of consistency category
Keywords
Full Text:
PDFReferences
Aghekyan, R. (2020). Validation of the SIEVEA instrument using the Rasch analysis. International Journal of Educational Research, 103, 101619.
Ariffin, S. R., Omar, B., Isa, A., & Sharif, S. (2010). Validity and reliability multiple intelligent item using rasch measurement model. Procedia-Social and Behavioral Sciences, 9, 729-733.
Arnold, J. C., Boone, W. J., Kremer, K., & Mayer, J. (2018). Assessment of competencies in scientific inquiry through the application of Rasch measurement techniques. Education Sciences, 8(4), 184.
Bond, T., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.
Boone, W. J. (2016). Rasch analysis for instrument development: why, when, and how?. CBE—Life Sciences Education, 15(4), rm4.
Boone, W. J. (2020). Rasch Basics for the Novice. In Rasch Measurement (pp. 9-30). Springer, Singapore.
Boone, W. J., & Noltemeyer, A. (2017). Rasch analysis: A primer for school psychology researchers and practitioners. Cogent Education, 4(1), 1416898.
Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multipleâ€choice tests. Science Education, 90(2), 253-269.
Boone, W. J., Townsend, J. S., & Staver, J. (2011). Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts: An exemplar utilizing STEBI selfâ€efficacy data. Science Education, 95(2), 258-280.
Brann, K. L., Boone, W. J., Splett, J. W., Clemons, C., & Bidwell, S. L. (2020). Development of the School Mental Health Self-Efficacy Teacher Survey Using Rasch Analysis. Journal of Psychoeducational Assessment, 0734282920947504.
Cavilla, D. (2017). The effects of student reflection on academic performance and motivation. SAGE Open, 7(3), 2158244017733790.
Chao, J., Siu, A. M., Leung, O., Lo, A., Chu, M., Lee, W. K., ... & Chien, C. W. (2019). Chinese version of the Recovery Self-Assessment scale: psychometric evidence from Rasch analysis and reliability estimates. Journal of Mental Health, 28(2), 206-212.
Chen, F. F., Chen, S. Y., & Pai, H. C. (2019). Self-reflection and critical thinking: the influence of professional qualifications on registered nurses. Contemporary nurse, 55(1), 59-70.
Chen, Y. H., & Lin, Y. J. (2018). Validation of the short self-regulation questionnaire for Taiwanese college students (TSSRQ). Frontiers in Psychology, 9, 259.
Cupani, M., Zamparella, T. C., Piumatti, G., & Vinculado, G. (2017). Development of an Item Bank for the Assessment of Knowledge on Biology in Argentine University Students. Journal of applied measurement, 18(3), 360-369.
Diefes-Dux, H. A., & Castro, L. M. C. (2018, October). Student reflection to improve access to standards-based grading feedback. In 2018 IEEE Frontiers in Education Conference (FIE) (pp. 1-9). IEEE.
Fiedler, D., Sbeglia, G. C., Nehm, R. H., & Harms, U. (2019). How strongly does statistical reasoning influence knowledge and acceptance of evolution?. Journal of Research in Science Teaching, 56(9), 1183-1206.
Fisher Jr, W. P. (2017). Suggestions for rethinking validation. Measurement: Interdisciplinary Research and Perspectives, 15(2), 86-90.
Garzón Umerenkova, A., de la Fuente Arias, J., MartÃnez-Vicente, J. M., Zapata Sevillano, L., Pichardo, M. C., & GarcÃa-Berbén, A. B. (2017). Validation of the Spanish short self-regulation questionnaire (SSSRQ) through Rasch analysis. Frontiers in psychology, 8, 276.
Großschedl, J., Mahler, D., & Harms, U. (2018). Construction and evaluation of an instrument to measure content knowledge in biology: The CK-IBI. Education Sciences, 8(3), 145.
Harpe, S. E. (2015). How to analyze Likert and other rating scale data. Currents in pharmacy teaching and learning, 7(6), 836-850.
He, P., Liu, X., Zheng, C., & Jia, M. (2016). Using Rasch measurement to validate an instrument for measuring the quality of classroom teaching in secondary chemistry lessons. Chemistry Education Research and Practice, 17(2), 381-393.
Herrmann-Abell, C. F., Hardcastle, J., & DeBoer, G. E. (2018). Using Rasch to Develop and Validate an Assessment of Students' Progress on the Energy Concept. Grantee Submission.
Kudiya, K., Sumintono, B., Sabana, S., & Sachari, A. (2018). Batik Artisans’ Judgment of Batik Wax Quality and Its Criteria: An Application of the Many-Facets Rasch Model. In Pacific Rim Objective Measurement Symposium (PROMS) 2016 Conference Proceedings (pp. 27-37). Springer, Singapore.
Lambri, A., Mahamod, Z., Zakaria, N., Baharum, H., & Hanapi, Z. (2019). Validity and Reliability of the Malay Social Traits Instrument Using Rasch Measurement Model. International Journal of Academic Research in Business and Social Sciences, 9(7), 225–232.
Lew, M. D., & Schmidt, H. G. (2011). Self-reflection and academic performance: is there a relationship?. Advances in Health Sciences Education, 16(4), 529.
Linacre, J. M. (2012). A user’s guide to WINSTEPS MINISTEP. Rasch model computer programs. Beaverton, Oregon: Winsteps. com.
Maat, S. M., & Rosli, M. K. (2016). The Rasch model analysis for statistical anxiety rating scale (STARS). Creative Education, 7(18), 2820.
Maseko, J., Luneta, K., & Long, C. (2019). Towards validation of a rational number instrument: An application of Rasch measurement theory. Pythagoras, 40(1), 441.
Mayes, R., Rittschof, K., Dauer, J., & Gallant, B. (2019). Quantitative modelling biology undergraduate assessment. Letters in Biomathematics, 6(1), 1-27.
Misbah, M., Dewantara, D., Hasan, S. M., & Annur, S. (2018). The development of student worksheet by using Guided Inquiry Learning Model to train student's scientific attitude. Unnes Science Education Journal, 7(1).
Mohamad, M. M., Sulaiman, N. L., Sern, L. C., & Salleh, K. M. (2015). Measuring the validity and reliability of research instruments. Procedia-Social and Behavioral Sciences, 204, 164-171.
Park, M., & Liu, X. (2019). An investigation of item difficulties in energy aspects across biology, chemistry, environmental science, and physics. Research in Science Education, 1-18.
Peeters, M. J., & Martin, B. A. (2017). Validation of learning assessments: A primer. Currents in Pharmacy Teaching and Learning, 9(5), 925-933.
Pichardo, M. C., Cano, F., Garzón-Umerenkova, A., de la Fuente, J., Peralta-Sánchez, F. J., & Amate-Romera, J. (2018). Self-regulation questionnaire (SRQ) in Spanish adolescents: factor structure and rasch analysis. Frontiers in psychology, 9, 1370.
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111.
Pleasence, P., & Balmer, N. J. (2019). Development of a general legal confidence scale: A first implementation of the Rasch measurement model in empirical legal studies. Journal of Empirical Legal Studies, 16(1), 143-174.
Sabudin, S., Mansor, A. N., Meerah, S. M., & Muhammad, A. (2018). Validity and Reliability of Students’ Science and Technology Culture Instrument (BST-M) using Rasch Measurement Model. International Journal of Academic Research in Business and Social Sciences, 8(5), 986-995.
Susongko, P. (2016). Validation of science achievement test with the rasch model. Jurnal Pendidikan IPA Indonesia, 5(2), 268-277.
Upegui-Arango, L. D., Forkmann, T., Nielsen, T., Hallensleben, N., Glaesmer, H., Spangenberg, L., ... & Boecker, M. (2020). Psychometric evaluation of the Interpersonal Needs Questionnaire (INQ) using item analysis according to the Rasch model. Plos one, 15(8), e0232030.
Van Zile-Tamsen, C. (2017). Using Rasch analysis to inform rating scale development. Research in Higher Education, 58(8), 922-933.
Widhiarso, W., & Sumintono, B. (2016). Examining response aberrance as a cause of outliers in statistical analysis. Personality and Individual Differences, 98, 11-15.
Yan, Z. (2020). Self-assessment in the process of self-regulated learning and its relationship with academic achievement. Assessment & Evaluation in Higher Education, 45(2), 224-238.
Yan, Z., Brown, G. T., Lee, J. C. K., & Qiu, X. L. (2020a). Student self-assessment: Why do they do it?. Educational Psychology, 40(4), 509-532.
Yan, Z., Chiu, M. M., & Ko, P. Y. (2020b). Effects of self-assessment diaries on academic achievement, self-regulation, and motivation. Assessment in Education: Principles, Policy & Practice, 27(5), 562-583.
Yang, Y., He, P., & Liu, X. (2018). Validation of an instrument for measuring students’ understanding of interdisciplinary science in grades 4-8 over multiple semesters: A Rasch measurement study. International Journal of Science and Mathematics Education, 16(4), 639-654.
Yasin, R. M., Yunus, F. A. N., Rus, R. C., Ahmad, A., & Rahim, M. B. (2015). Validity and reliability learning transfer item using rasch measurement model. Procedia-Social and Behavioral Sciences, 204, 212-217.
Yuhanna, W. L., & Retno, R. S. (2016). The learning of science basic concept by using scientifiq inquiry to improve student’s thinking, working, and scientific attitude abilities. JPBI (Jurnal Pendidikan Biologi Indonesia), 2(1), 1-9.
Refbacks
- There are currently no refbacks.
