Analysis of Items for Measuring Technological Literacy Using the Rasch Model
DOI:
https://doi.org/10.15294/04k5q459Keywords:
analysis of test items, technological literacy; rasch modelAbstract
This research aims to describe the results of the analysis of the quality of the test items for technological literacy skills using the Rasch model from the aspects of validity, reliability, and level of difficulty of the questions. This research uses a descriptive quantitative approach, with construction design and validation. Data was obtained through a test method in the form of 15 multiple-choice questions and a questionnaire with 40 questions. The test was carried out on 6th-semester physics students with an integrated science course totaling 30 students. Data analysis was carried out by describing the results of data processing using the Rasch Model with Winsteps software. The research results showed that the quality of the items on the instrument measuring technological literacy in the validity aspect contained 7 valid questions, and 6 invalid questions regarding knowledge. Meanwhile, 36 questions about attitude measurement were valid and 4 questions were invalid. In the reliability aspect for knowledge questions, the reliability item value is 0.93 in the very good category. Meanwhile, in the attitude questionnaire, the item reliability value was 0.98 in the special category. In terms of the level of difficulty of the questions, for knowledge questions, there are 2 easy questions, 5 medium, 5 difficult, and 3 very difficult. In the very difficult question category, only students with high abilities can answer. For attitude questions, there are 7 easy questions, 14 medium, 9 difficult, and 10 very difficult questions. It can be concluded that this technological literacy test item is suitable for use in assessing technological literacy competency so that it can contribute to measuring technological literacy which has not been done much.
References
Ahmadi, M. H., Ghazvini, M., Sadeghzadeh, M., Alhuyi Nazari, M., Kumar, R., Naeimi, A., & Ming, T. (2018). Solar power technology for electricity generation: A critical review. Energy Science and Engineering, 6(5), 340–361.
Alfarisa, F., & Purnama, D. N. (2019). Analisis Butir Soal Ulangan Akhir Semester Mata Pelajaran Ekonomi SMA Menggunakan RASCH Model, 11(2), 366-374.
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221.
Andrade, C. (2018). Internal, External, and Ecological Validity in Research Design, Conduct, and Evaluation. Indian Journal of Psychological Medicine, 40(5), 498–499.
Arikunto, S. (2012). Dasar-dasar evaluasi Pendidikan (Restu Damayanti (ed.); Edisi ke-2). Bumi Aksara.
Arikunto, S. (2017). Pengembangan Instrumen Penelitian dan Penilaian Program. Yokyakarta: Pustaka Pelajar.
Asrizal, A., Amran, A., Ananda, A., & Festiyed, F. (2018). Effectiveness of Adaptive Contextual Learning Model of Integrated Science by Integrating Digital Age Literacy on Grade VIII Students. IOP Conference Series: Materials Science and Engineering, 335(1).
Asunda, P. A. (2012). Standards for technological literacy and STEM education delivery through career and technical education programs. Journal of Technology Education, 23(2), 44–60.
Ayub, M. R. S. S. N., Istiyono, E., Munadi, S., Permadi, C., Pattiserlihun, A., & Sudjito, D. N. (2020). Analisa Penilaian Soal Fisika Menggunakan Model Rasch Dengan Program R. Jurnal Sains Dan Edukasi Sains, 3(2), 46–52.
Azizah, A., & Wahyuningsih, S. (2020). Penggunaan Model Rasch Untuk Analisis Instrumen Tes Pada Mata Kuliah Matematika Aktuaria. JUPITEK: Jurnal Pendidikan Matematika, 3(1), 45–50.
Azwar, S. (2019). Penyusunan Skala Psikologi. Yogyakarta: Pustaka Pelajar.
Balsalobre-Fernández, C., Kuzdub, M., Poveda-Ortiz, P., & Campo-Vecino, J. del. (2016). Validity and Reliability of the PUSH Wearable Device to Measure Movement Velocity During the Back Squat Exercise. Journal of Strength and Conditioning Research, 30(7), 1968–1974.
Borges, N. R., & Driller, M. W. (2016). Wearable Lactate Threshold Predicting Device is Valid and Reliable in Runners. Journal of Strength and Conditioning Research, 30(8), 2212–2218.
Creswell, J., & Poth, C. (2018). Qualitative Inguiry Research Design: Choosing amoung five approaches (4th ed.). California : SAGE Publications, Inc., Thousand Oaks.
Diegel, O., Nordin, A., & Motte, D. (2019). Additive Manufacturing Technologies. Springer Series in Advanced Manufacturing, 19–39.
Erfan, M., Maulyda, M. A., Hidayati, V. R., Astria, F. P., & Ratu, T. (2020). Tes Klasik Dan Model Rasch. Indonesian Journal of Educational Research and Review, 3(1), 11–19.
Ferrando, P. J., & Lorenzo-Seva, U. (2018). Assessing the Quality and Appropriateness of Factor Solutions and Factor Score Estimates in Exploratory Item Factor Analysis. Educational and Psychological Measurement, 78(5), 762–780.
Ferrando, P. J., Lorenzo-Seva, U., Hernández-Dorado, A., & Muñiz, J. (2022). Decalogue for the Factor Analysis of Test Items. Psicothema, 34(1), 7–17.
Fuller, D., Colwell, E., Low, J., Orychock, K., Ann Tobin, M., Simango, B., Buote, R., van Heerden, D., Luan, H., Cullen, K., Slade, L., & Taylor, N. G. A. (2020). Reliability and Validity of Commercially Available Wearable Devices for Measuring Steps, Energy Expenditure, and Heart Rate: Systematic Review. JMIR MHealth and UHealth, 8(9), 1–23.
Garmire, E., & Pearson, G. (2006). Tech tally: Approaches to assessing technological literacy. In Tech Tally: Approaches to Assessing Technological Literacy.
Groth, L. A., Dunlap, K. L., & Kidd, J. K. (2007). Becoming technologically literate through technology integration in PK‐12 preservice literacy courses: Three case studies. Reading Research and Instruction, 46(4), 363–386.
Hamdu, G., Fuadi, F. N., Yulianto, A., & Akhirani, Y. S. (2020). Items Quality Analysis Using Rasch Model To Measure Elementary School Students’ Critical Thinking Skill On Stem Learning. JPI (Jurnal Pendidikan Indonesia), 9(1), 61.
Hardianti, H., Liliawati, W., & Tayubi, Y. R. (2021). Karakteristik Tes Kemampuan Berpikir Kritis Siswa SMA Pada Materi Momentum dan Impuls: Perbandingan Classical Theory Test (CTT) dan model Rasch. WaPFi (Wahana Pendidikan Fisika), 8(1), 21-28.
Hohlfeld, T. N., Ritzhaupt, A. D., & Barron, A. E. (2010). Development and Validation of the Student Tool for Technology Literacy (ST 2 L). Journal of Research on Technology in Education, 42(4), 361–389.
Ismail, F. K. M., & Zubairi, A. M. B. (2021). Item Objective Congruence Analysis for Multidimensional Items Content Validation of a Reading Test in Sri Lankan University. English Language Teaching, 15(1), 106.
Jumini, S., Madnasri, S., Cahyono, E., & Parmin, P. (2023). Analisis Kualitas Butir Soal Pengukuran Literasi Sains Melalui Teori Tes Klasik Dan Rasch Model. Prosiding Seminar Nasional Pascasarjana UNNES, 758–765.
Keinänen, M., Ursin, J., & Nissinen, K. (2018). How to measure students’ innovation competences in higher education: Evaluation of an assessment tool in authentic learning environments. Studies in Educational Evaluation, 58, 30–36.
Nahijah, A. R., Serevina, V., & Delina, M. (2018). The Development of High Order Thinking Skills (HOTS) Assessment Instrument for Temperature and Heat Learning. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 4(1), 19-26.
Neumeyer, X., Santos, S. C., & Morris, M. H. (2021). Overcoming Barriers to Technology Adoption When Fostering Entrepreneurship Among the Poor: The Role of Technology and Digital Literacy. IEEE Transactions on Engineering Management, 68(6), 1605–1618.
Peeters, M. J., & Augustine, J. M. (2023). Using Rasch measurement for instrument rating scale refinement. Currents in Pharmacy Teaching and Learning, 15(1), 110–118.
Peeters, M. J., & Harpe, S. E. (2022). Contemporary conceptualization of measurement validity. In Contemporary Research Methods in Pharmacy and Health Services (pp. 575–582). Elsevier.
Pfeifer, K., Ergal, İ., Koller, M., Basen, M., Schuster, B., & Rittmann, S. K.-M. R. (2021). Archaea Biotechnology. Biotechnology Advances, 47, 107668.
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 20111.
Purniasari, L., Masykuri, M., & Ariani, S. R. D. (2021). Analisis Butir Soal Ujian Sekolah Mata Pelajaran Kimia Sma N 1 Kutowinangun Tahun Pelajaran 2019/2020 Menggunakan Model Iteman Dan Rasch. Jurnal Pendidikan Kimia, 10(2), 205–214.
Putranta, H., Jumadi., J., & Wilujeng, I. (2019). Physics learning by PhET simulation-assisted using problem based learning (PBL) model to improve students’ critical thinking skills in work and energy chapters in MAN 3 Sleman. In Asia-Pacific Forum on Science Learning and Teaching (Vol. 20, No. 1, pp. 1-44).
Raharjo, S., Kaniawati, I., & Suwarma, I. R. (2014). Technology and Engineering Literacy Profile of Senior High School Students ’ on Understanding The Newton Laws ’. 3–8.
Rahmawati, T., Mulyaningsih, T., Nahadi, N., Suhanda, H., Lee, W. K., Aziz, H. A., & Anwar, S. (2023). Electronic Portfolio Assessment Instruments in Improving Students’ Creative Thinking Skills. Jurnal Pendidikan IPA Indonesia, 12(4), 598–610.
Schaufeli, W. B., Desart, S., & De Witte, H. (2020). Burnout Assessment Tool (BAT)—Development, Validity, and Reliability. International Journal of Environmental Research and Public Health, 17(24), 9495.
Silvia, P. J., Rodriguez, R. M., Beaty, R. E., Frith, E., Kaufman, J. C., Loprinzi, P., & Reiter-Palmon, R. (2021). Measuring everyday creativity: A Rasch model analysis of the Biographical Inventory of Creative Behaviors (BICB) scale. Thinking Skills and Creativity, 39, 100797.
Sudaryono, S., Margono, G., & Rahayu, W. (2013). Pengembangan Instrumen Penelitian Pendidikan. Yogyakarta: Graha Ilmu.
Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial (edisi revisi). Trim Komunikata Publishing House.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan RASCH Pada Assessment Pendidikan (C. T. Komunikata (ed.)).
Supriyadi, T., Saptani, E., Rukmana, A., Suherman, A., Alif, M. N., & Rahminawati, N. (2020). Students’ technological literacy to improve academic writing and publication quality. Universal Journal of Educational Research, 8(11B), 6022–6035.
Susanta, A., & Susanto, E. (2021). The development of test instruments to support students’ ability on mathematical learning evaluation course. Journal of Physics: Conference Series, 1731(1).
Taher, M., & Abtaria, Y. (2017). Efektifitas Pembelajaran Creative Problem Solving Berbasis Eksperimen dalam Pembelajaran Fisika untuk Melatihkan Literasi Teknologi Siswa. Gravity, 3(2), 148–157.
Triana, A. J., Gusdorf, R. E., Shah, K. P., & Horst, S. N. (2020). Technology Literacy as a Barrier to Telehealth During COVID-19. Telemedicine and E-Health, 26(9), 1118–1119.
Upadhayaya, P. R. (2023). Information Communication Technology in Education: Bringing Innovation in Classroom. Ganeshman Darpan, 8(1), 96–110.
Wensveen, J. (2023). Air Transportation: A Global Management Perspective. Routledge.
Widoyoko, E. P. (2012). Teknik Penyusunan Instrumen Penelitian. Yogyakarta: Pustaka Pelajar.
Willis, E., Gundacker, C., Harris, M., & Mameledzija, M. (2019). Improving immunization and health literacy through a community-based approach enhanced by technology. Information Services and Use, 39(1–2), 23–36.
Yan, Q., Dong, H., Su, J., Han, J., Song, B., Wei, Q., & Shi, Y. (2018). A Review of 3D Printing Technology for Medical Applications. Engineering, 4(5), 729–742.
Yu-Te, W., Yuan-Tai, C., Shun, L. Y., & Kai-Fu, Y. (2014). A survey of Energy Technology literacy level for senior high school students in Taiwan. In Proceedings of International Conference on Engineering and Computer Education (Vol. 6).
Zainurrisalah, T. F., Suwarma, I. R., & Jauhari, A. (2018). Mengukur Kemampuan Literasi Teknologi dan Rekayasa (Engineering) Melalui Penerapan Pembelajaran STEM dalam Fisika. Prosiding Seminar Nasional Fisika (E-Journal), 131–135.
