Quality of Four-Tier Diagnostic Test on Wave and Vibration Materials: An Empirical Study Using Rasch Modeling
(1) Physics Education Study Program, Universitas Ahmad Dahlan, Indonesia
(2) Physics Education Study Program, Universitas Ahmad Dahlan, Indonesia
(3) Physics Education Study Program, Universitas Ahmad Dahlan, Indonesia
(4) Department of Physics, Universiti Pendidikan Sultan Idris, Malaysia
Abstract
Not all teachers have the ability to develop diagnostic instruments for misconception. Meanwhile, there is an urgent need to diagnose student misconceptions. Therefore, this research aims to carry out the process of adapting misconception-diagnostic instruments to wave and vibration materials. This study used a cross-sectional quantitative survey method. The adaptation process was conducted with 306 state high school students taking wave and vibration classes. Respondents were selected using convenience sampling techniques. 4WADI (Four-tier Wave and vibration Diagnostic Instrument) was translated into Indonesian, consulted with language experts, and assessed by material and evaluation experts to produce an instrument appropriate to the Indonesian cultural context. Before the empirical test was carried out, I-4WADI (Indonesian et al. and vibration Diagnostic Instrument) was tested for readability on 35 students. As a result, the I-4WADI has excellent readability. The validation results of six experts were analyzed using the Aiken V technique. A total of 25 items were used for expert validation. Empirical validation data were analyzed using the Rasch model. Review empirical validation from the aspects of item fit and unidimensionality. The analysis results show that I-4WADI has good quality based on validity and reliability. The infit MnSq values ranged from 0.81 to 1.40, and the outfit MnSq values ranged from 0.71 to 1.36. The instrument reliability value is 0.98. So, I-4WADI can be applied to high school students in Indonesia. These findings have practical implications for teachers who wish to diagnose students' misconceptions about waves and vibration.
Keywords
Full Text:
PDFReferences
Adams, D., Tan, M. H. J., & Sumintono, B. (2021). Students’ readiness for blended learning in a leading Malaysian private higher education institution. Interactive Technology and Smart Education, 18(4), 515–534. Retrieved from https://doi.org/10.1108/ITSE-03-2020-0032
Assimi, E., Janati I, R., Zerhane, R., & Boubih, S. (2022). The use of a three-tier diagnostic test to investigate conceptions related to cell biology concepts among pre-service teachers of life and earth sciences. Journal of Biological Education, 1–28. Retrieved from https://doi.org/10.1080/00219266.2022.2134175
Aykutlu, I., Ensari, Ö., & Bayrak, C. (2023). Prospective teachers’ conceptual understanding of the polarization of light. European Journal of Physics, 44(1), 015701. Retrieved from https://doi.org/10.1088/1361-6404/ac93cc
Bani-Salameh, H. N. (2017). How persistent are the misconceptions about force and motion held by college students? Physics Education, 52(1), 14003.
Brundage, M. J., & Singh, C. (2023). Development and validation of a conceptual multiple-choice survey instrument to assess student understanding of introductory thermodynamics. Physical Review Physics Education Research, 19(2), 020112. Retrieved from https://doi.org/10.1103/PhysRevPhysEducRes.19.020112
Caleon, I. S., & Subramaniam, R. (2010). Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions. Research in Science Education, 40(3), 313–337. Retrieved from https://doi.org/10.1007/s11165-009-9122-4
Caycho-Rodríguez, T., Vilca, L. W., Cervigni, M., Gallegos, M., Martino, P., Portillo, N., & Burgos Videla, C. (2022). Fear of COVID-19 scale: Validity, reliability and factorial invariance in Argentina’s general population. Death Studies, 46(3), 543–552. doi:10.1080/07481187.2020.1836071
Çelikkanlı, N. Ö., & Kızılcık, H. Ş. (2022). A Review of Studies About Four-Tier Diagnostic Tests in Physics Education. Journal of Turkish Science Education, 19(4), 1291–1311. Retrieved from https://doi.org/10.36681/tused.2022.175
Chen, W.-H., Lenderking, W., Jin, Y., Wyrwich, K. W., Gelhorn, H., & Revicki, D. A. (2014). Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank data. Quality of Life Research, 23(2), 485–493. Retrieved from https://doi.org/10.1007/s11136-013-0487-5
Chou, Y.-T., & Wang, W.-C. (2010). Checking Dimensionality in Item Response Models with Principal Component Analysis on Standardized Residuals. Educational and Psychological Measurement, 70(5), 717–731. Retrieved from https://doi.org/10.1177/0013164410379322
Cordier, R., Speyer, R., Schindler, A., Michou, E., Heijnen, B. J., Baijens, L., & Joosten, A. V. (2018). Using Rasch Analysis to Evaluate the Reliability and Validity of the Swallowing Quality of Life Questionnaire: An Item Response Theory Approach. Dysphagia, 33(4), 441–456. Retrieved from https://doi.org/10.1007/s00455-017-9873-4
Dwiastuti, E., Sukarelawan, M. I., & Sriyanto, S. (2022). Learning Management System Acceptance Rate Among Vocational School Physics’ Student. JIPFRI (Jurnal Inovasi Pendidikan Fisika Dan Riset Ilmiah), 6(1), 10–18. Retrieved from https://doi.org/10.30599/jipfri.v6i1.1183
Guo, H., Lu, R., Johnson, M. S., & McCaffrey, D. F. (2022). Alternative Methods for Item Parameter Estimation: From CTT to IRT. ETS Research Report Series, 2022(1), 1–16. Retrieved from https://doi.org/10.1002/ets2.12355
Ha, H. T. (2021). A Rasch-based validation of the Vietnamese version of the Listening Vocabulary Levels Test. Language Testing in Asia, 11(1), 16. Retrieved from https://doi.org/10.1186/s40468-021-00132-7
Haidar, D. A., Yuliati, L., & Handayanto, S. K. (2020). The effect of inquiry learning with scaffolding on misconception of light material among fourth-grade students. Jurnal Pendidikan IPA Indonesia, 9(4), 540–553. Retrieved from https://doi.org/10.15294/jpii.v9i4.22973
Han, H. (2022). The effectiveness of weighted least squares means and variance adjusted based fit indices in assessing local dependence of the rasch model: Comparison with principal component analysis of residuals. PLOS ONE, 17(9), e0271992. Retrieved from https://doi.org/10.1371/journal.pone.0271992
Kaniawati, I., Fratiwi, N. J., Danawan, A., Suyana, I., Samsudin, A., & Suhendi, E. (2019). Analyzing students’ misconceptions about Newton’s Laws through Four-Tier Newtonian Test (FTNT). Journal of Turkish Science Education, 16(1), 110–122. Retrieved from https://doi.org/10.12973/tused.10269a
Larasati, P. E., Supahar, & Yunanta, D. R. A. (2020). Validity and reliability estimation of assessment ability instrument for data literacy on high school physics material. Journal of Physics: Conference Series, 1440(1), 012020. Retrieved from https://doi.org/10.1088/1742-6596/1440/1/012020
Lemmer, M., Kriek, J., & Erasmus, B. (2020). Analysis of Students’ Conceptions of Basic Magnetism from a Complex Systems Perspective. Research in Science Education, 50(2), 375–392. Retrieved from https://doi.org/10.1007/s11165-018-9693-z
Li, D., Zong, X., Huang, Q., Wu, F., Huang, Y., Ge, Y., … Yuan, C. (2023). Validation of the simplified Chinese version of PROMIS Parent Proxy-25 Profile in parents of children with cancer. Journal of Pediatric Nursing, in press. Retrieved from https://doi.org/10.1016/j.pedn.2023.05.016
Lim, H. L., & Poo, Y. P. (2021). Diagnostic test to assess misconceptions on photosynthesis and plant respiration: Is it valid and reliable? Jurnal Pendidikan IPA Indonesia, 10(2), 241–252. Retrieved from https://doi.org/10.15294/jpii.v10i2.26944
Ling Lee, W., Chinna, K., & Sumintono, B. (2020). Psychometrics assessment of HeartQoL questionnaire: A rasch analysis. European Journal of Preventive Cardiology, 2047487320902322. Retrieved from https://doi.org/10.1177/2047487320902322
Liu, X., & Boone, W. J. (2023). Introduction to Advances in Applications of Rasch Measurement in Science Education. In X. Liu & W. J. Boone (Eds.), Advances in Applications of Rasch Measurement in Science Education. Contemporary Trends and Issues in Science Education (pp. 1–18). Springer. Retrieved from https://doi.org/10.1007/978-3-031-28776-3_1
Luce, M. R., & Callanan, M. A. (2020). Family Conversations About Heat and Temperature: Implications for Children’s Learning. Frontiers in Psychology, 11(August), 1–18. Retrieved from https://doi.org/10.3389/fpsyg.2020.01718
Maknun, J., & Marwiah, M. (2022). Remediation of Misconceptions Vocational High School Students on the Concept of Static Fluids using the Conceptual Change Model. Journal of Technical Education and Training, 14(2). Retrieved from https://doi.org/10.30880/jtet.2022.14.02.005
Matheny, L. M., & Clanton, T. O. (2020). Rasch Analysis of Reliability and Validity of Scores From the Foot and Ankle Ability Measure (FAAM). Foot and Ankle International, 41(2), 229–236. Retrieved from https://doi.org/10.1177/1071100719884554
Monaco, E., Pisanu, G., Carrozzo, A., Giuliani, A., Conteduca, J., Oliviero, M. & Ferretti, A. (2022). Translation, cross-cultural adaptation, and validation of the Italian version of the anterior cruciate ligament–return to sport after injury (ACL-RSI) scale and its integration into the K-STARTS test. Journal of Orthopaedics and Traumatology, 23(1), 1–8. Retrieved from https://doi.org/10.1186/s10195-021-00622-7
Muñiz, J., Elosua, P., Padilla, J.-L., & Hambleton, R. K. (2016). Test Adaptation Standards for Cross-Lingual Assessment. In C. S. Wells, M. Faulkner-Bond, & E. Hambleton (Eds.), Educational Measurement: From Foundations to Future (pp. 291–304). The Guilford Press.
Negoro, R. A., & Karina, V. (2019). Development of a Four-Tier Diagnostic Test for Misconception of Oscillation and Waves. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 5(2), 69–76. Retrieved from https://doi.org/10.21009/1.05201
Palacios-Espinosa, X., Pedraza, R. S., Gómez-Carvajal, A.-M., Botero-Meneses, J. S., Escallón, D. M., & Leal, D. A. (2023). Psychometric Properties of the Death Anxiety Scale for Adult Chronic Patients. Trends in Psychiatry and Psychotherapy, in press. Retrieved from https://doi.org/10.47626/2237-6089-2023-0630
Rusmansyah, R., & Almubarak, A. (2020). Students’ Cognitive Analysis using Rasch Modeling as an Assessment for Planning of Strategies in Chemistry Learning. JKPK (Jurnal Kimia Dan Pendidikan Kimia), 5(3), 222. Retrieved from https://doi.org/10.20961/jkpk.v5i3.40936
Scoulas, J. M., Aksu Dunya, B., & De Groote, S. L. (2021). Validating students’ library experience survey using rasch model. Library & Information Science Research, 43(1), 101071. Retrieved from https://doi.org/10.1016/j.lisr.2021.101071
Simpelaere, I. S., Van Nuffelen, G., De Bodt, M., Vanderwegen, J., & Hansen, T. (2017). Validation of the Dutch version of the Swallowing Quality-of-Life Questionnaire (DSWAL-QoL) and the adjusted DSWAL-QoL (aDSWAL-QoL) using item analysis with the Rasch model: a pilot study. Health and Quality of Life Outcomes, 15(1), 66. Retrieved from https://doi.org/10.1186/s12955-017-0639-3
Soeharto, S., & Csapó, B. (2022). Exploring Indonesian Student Misconceptions in Science Concepts. Heliyon, 8(9), e10720. Retrieved from https://doi.org/10.1016/j.heliyon.2022.e10720
Somroob, S., & Wattanakasiwich, P. (2017). Investigating student understanding of simple harmonic motion. Journal of Physics: Conference Series, 901, 012123. Retrieved from https://doi.org/10.1088/1742-6596/901/1/012123
Stefanou, M., Stylos, G., Georgopoulos, K., & Kotsis, K. (2023). Primary Preservice Teachers’ Misconceptions and Reasoning of Thermal Concepts in Everyday Contexts. The International Journal of Learning in Higher Education, 31(1), 127–157. Retrieved from https://doi.org/10.18848/2327-7955/CGP/v31i01/127-157
Suhendi, H. Y., & Ardiansyah, R. (2021). Development of HATRADI : a four tier test for diagnostic student misconception in heat transfer concept. Journal of Physics: Conference Series, 1918(2). Retrieved from https://doi.org/10.1088/1742-6596/1918/2/022015
Sukarelawan, M. I., Jumadi, J., Kuswanto, H., Soeharto, S., & Hikmah, F. N. (2021). Rasch Analysis to Evaluate the Psychometric Properties of Junior Metacognitive Awareness Inventory in the Indonesian Context. Jurnal Pendidikan IPA Indonesia, 10(4), 486–495. Retrieved from https://doi.org/10.15294/jpii.v10i4.27114
Sukarelawan, M. I., Jumadi, J., Kuswanto, H., & Thohir, M. A. (2021). The Indonesian Version of the Physics Metacognition Inventory: Confirmatory Factor Analysis and Rasch Model. European Journal of Educational Research, 10(4), 2133–2144. doi:10.12973/eu-jer.10.4.2133
Sukarelawan, M. I., Puspitasari, A. D., Rahmatika, Z. ’Amalia, Dennis, D., Ishafit, I., Hikmah, F. N., & Sulistyo, E. N. (2022). Online Learning Using Google Incorporated for Student High School: Mapping Motivation Using Rasch Model in Physics Learning. Berkala Ilmiah Pendidikan Fisika, 10(2), 216–224. Retrieved from https://doi.org/10.20527/bipf.v10i2.12209
Sukarelawan, M. I., Puspitasari, A. D., Sulisworo, D., Kuswanto, H., & Jumadi. (2022). A shift in the conceptual understanding of physics students through the wright map. Jurnal Pendidikan Dan Pengajaran, 55(1), 127–141. Retrieved from https://doi.org/10.23887/jpp.v55i1.37155
Sukarelawan, M. I., Sriyanto, S., Puspitasari, A. D., Sulisworo, D., & Hikmah, U. N. (2021). Four-Tier Heat and Temperature Diagnostic Test (4T-HTDT) to Identify Student Misconceptions. JIPFRI (Jurnal Inovasi Pendidikan Fisika Dan Riset Ilmiah), 5(1), 1–8. Retrieved from https://doi.org/10.30599/jipfri.v5i1.856
Sumintono, B. (2015). Pemodelan Rasch pada Asesmen Pendidikan: Suatu Pengantar. Konferensi Guru Dan Dosen Nasional (KGDN) 2015, (November), 1–14. Retrieved from http://eprints.um.edu.my/id/eprint/15603%0A
Sumintono, B., & Widhiarso, W. (2014). Aplikasi model rasch untuk penelitian ilmu-ilmu sosial [Rasch model application for social sciences research]. Cimahi: Trim Komunikata Publishing House.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan Rasch pada Asesmen Pendidikan. Cimahi: Trim Komunikata.
Syahrial, S., Ilmah, M., Yahmin, Y., Munzil, M., & Muntholib, M. (2023). Remediation of chemistry teachers’ misconceptions about covalent bonding using cognitive conflict interviews: A case study. Journal of the Serbian Chemical Society, 88(2), 211–221. Retrieved from https://doi.org/10.2298/JSC220117073S
Yuhanna, W. L., Al Muhdhar, M. H. I., Gofur, A., & Hassan, Z. (2021). Self-reflection assessment in vertebrate zoology (Sravz) using rasch analysis. Jurnal Pendidikan IPA Indonesia, 10(1), 35–47. Retrieved from https://doi.org/10.15294/jpii.v10i1.25603
Zhao, C., Zhang, S., Cui, H., Hu, W., & Dai, G. (2023). Middle school students’ alternative conceptions about the human blood circulatory system using four-tier multiple-choice tests. Journal of Biological Education, 57(1), 51–67. Retrieved from https://doi.org/10.1080/00219266.2021.1877777
Zmnako, S. S. F., & Chalabi, Y. I. (2019). Cross-cultural adaptation, reliability, and validity of the Vertigo symptom scale-short form in the central Kurdish dialect. Health and Quality of Life Outcomes, 17(1), 1–12. Retrieved from https://doi.org/10.1186/s12955-019-1168-z
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution 4.0 International License