Validation and Implementation of 3-Dimensional Scientific Literacy Test (Lisa3D Test): Measuring Scientific Literacy for Senior High School Students based on Scientific Reasoning, Scientific Inquiry, and Nature of Science
DOI:
https://doi.org/10.15294/591rx526Keywords:
inquiry, Lisa3D test, nature of science, reasoning, scientific, validationAbstract
This research aims to validate and implement an instrument to measure the scientific literacy of high school science program students, including the abilities of Scientific Reasoning, Scientific Inquiry, and Nature of Science (NOS), called the Lisa3D Test. These three instruments were validated using the Rasch model approach and presented as a digital application. This research method was a descriptive survey with a quantitative approach. The research started from the test design, validation, and implementation stages. The types of validity used included content, psychometric, construct, and external validity. For construct validity, the instrument was tested on 197 high school students, while the implementation stage involved 407 students. The research results show that the Lisa3D Test has fulfilled all aspects of validity. The Lisa3D test has been successfully implemented via the page www.lisatest.id with a cut of 56% for the Scientific Reasoning test, 50% for the Scientific Inquiry test, and 56.25% for the NOS test. This research produces a standard instrument to measure the scientific literacy of high school science program students based on three dimensions, namely Scientific Reasoning, Scientific Inquiry, and Nature of Science (NOS). The Lisa3D test can be implemented on a broader scale and can be an additional criterion for graduating high school students so that all graduates can master science comprehensively.
References
Adnan, A., Usman, U., & Bahri, A. (2021). Scientific literacy skills of students: Problem of biology teaching in junior high school in south sulawesi, indonesia. International Journal of Instruction, 14(3), 847-860.
Andrich, D., & Marais, I. (2019). A course in Rasch measurement theory. Measuring in the educational, social and health sciences, 41.
Anselmi, P., Vidotto, G., Bettinardi, O., & Bertolotti, G. (2015). Measurement of change in health status with Rasch models. Health and Quality of Life Outcomes, 13, 1-7.
Ardiyanti, Y., Suyanto, S., & Suryadarma, I. G. P. (2019, October). The role of students science literacy in Indonesia. In Journal of Physics: Conference Series (Vol. 1321, No. 3, p. 032085). IOP Publishing.
Arfiani, Y., Susongko, P., & Kusuma, M. (2023). Construct validity analysis with messick validity approach and rasch model application on scientific reasoning test items. THABIEA: JOURNAL OF NATURAL SCIENCE TEACHING, 6(1), 90-105.
Ariyanti, A. I. P., Ramli, M., & Prayitno, B. A. (2016). Preliminary Study on Developing Science Literacy Test for High School Students in Indone-sia. In International Conference on Teacher Training and Education. Sebelas Maret University.
Atta, H. B., & Aras, I. (2020). Developing an instrument for students scientific literacy. In Journal of Physics: Conference Series (Vol. 1422, No. 1, p. 012019). IOP Publishing.
Baldwin, P., Margolis, M. J., Clauser, B. E., Mee, J., & Winward, M. (2020). The choice of response probability in bookmark standard setting: an experimental study. Educational Measurement: Issues and Practice, 39(1), 37-44.
Bao, L., Xiao, Y., Koenig, K., & Han, J. (2018). Validity evaluation of the Lawson classroom test of scientific reasoning. Physical Review Physics Education Research, 14(2), 020106.
Bond, T., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.
Bordbar, S., & Alavi, S. M. (2021). Detecting gender-biased items in a high-stakes language proficiency test: using Rasch model measurement. International Journal of Quantitative Research in Education, 5(3), 277-310.
Briggs, D. C. (2019). Interpreting and visualizing the unit of measurement in the Rasch Model. Measurement, 146, 961-971.
Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. (2021). 21st century skills development through inquiry-based learning from theory to practice. Springer International Publishing.
Clauser, B. E., Baldwin, P., Margolis, M. J., Mee, J., & Winward, M. (2017). An experimental study of the internal consistency of judgments made in bookmark standard setting. Journal of Educational Measurement, 54(4), 481-497.
Creswell, J. W., & Creswell, J. D. (2017). Research de-sign: Qualitative, quantitative, and mixed methods approaches. Sage publications.
Dewi, C. A., Khery, Y., & Erna, M. (2019). An ethno-science study in chemistry learning to develop scientific literacy. Jurnal Pendidikan IPA Indonesia, 8(2), 279-287.
Erduran, S., Dagher, Z. R., Erduran, S., & Dagher, Z. R. (2014). Reconceptualizing nature of science for science education (pp. 1-18). Springer Netherlands.
Fakhriyah, F., Masfuah, S., Roysa, M., Rusilowati, A., & Rahayu, E. S. (2017). Student’s science literacy in the aspect of content science?. Jurnal Pendidikan IPA Indonesia, 6(1).
Filgona, J., Sakiyo, J., Gwany, D. M., & Okoronka, A. U. (2020). Motivation in learning. Asian Journal of Education and social studies, 10(4), 16-37.
Fives, H., Huebner, W., Birnbaum, A. S., & Nicolich, M. (2014). Developing a measure of scientific literacy for middle school students. Science Education, 98(4), 549-580.
Gall, M. D., Gall, J. P., & Borg, W. R. (2014). Applying educational research: How to read, do, and use research to solve problems of practice. Pearson Higher Ed.
Han, J. (2013). Scientific reasoning: Research, development, and assessment (Doctoral dissertation, The Ohio State University).Hanson, S. (2016). The assessment of scientific reasoning skills of high school science students: A standardized assessment instrument.
Howard, J. L., Bureau, J. S., Guay, F., Chong, J. X., & Ryan, R. M. (2021). Student motivation and associated outcomes: A meta-analysis from self-determination theory. Perspectives on Psychological Science, 16(6), 1300-1323.
Kind, P. E. R., & Osborne, J. (2017). Styles of scientific reasoning: a cultural rationale for science education?. Science Education, 101(1), 8-31.
Kluge, F., Gaßner, H., Hannink, J., Pasluosta, C., Klucken, J., & Eskofier, B. M. (2017). Towards mobile gait analysis: concurrent validity and test-retest reliability of an inertial measurement system for the assessment of spatiotemporal gait parameters. Sensors, 17(7), 1522.
Lawrence Neuman, W. (2019). Metodologi Penelitian sosial; Pendekatan Kualitatif dan Kuantitatif. Jakarta : PT Indeks
Lederman, J. S., Lederman, N. G., Bartos, S. A., Bartels, S. L., Meyer, A. A., & Schwartz, R. S. (2014). Meaningful assessment of learners’ understandings about scientific inquiry—The views about scientific inquiry (VASI) questionnaire. Journal of research in science teaching, 51(1), 65-83.
Lewis, D. M., Mitzel, H. C., Mercado, R. L., & Schulz, E. M. (2012). The bookmark standard setting procedure. In Setting performance standards (pp. 225-253). Routledge
Mayer, D., Sodian, B., Koerber, S., & Schwippert, K. (2014). Scientific reasoning in elementary school children: Assessment and relations with cognitive abilities. Learning and Instruction, 29, 43-55.
McComas, W. F. (2013). Nature of science in the science curriculum and in teacher education pro-grams in the United States. In International handbook of research in history, philosophy and science teaching (pp. 1993-2023). Dordrecht: Springer Netherlands.
McComas, W. F., & Nouri, N. (2016). The nature of science and the next generation science standards: Analysis and critique. Journal of Science Teacher Education, 27(5), 555-576.
McNamara, T. (2006). Validity in language testing: The challenge of Sam Messick’s legacy. Language Assessment Quarterly: An International Journal, 3(1), 31-51.
Molenda, M. (2015). In search of the elusive ADDIE model. Performance Improvement, 54(2), 40-42
Mulbar, U., & Bahri, A. (2021, February). Biology Science Literacy of Junior High School Students in South Sulawesi, Indonesia. In Journal of Physics: Conference Series (Vol. 1752, No. 1, p. 012084). IOP Publishing.
Muniroh, N., Rusilowati, A., & Isnaeni, W. (2022). Instrument Development of Science Literacy Assessment with Socio-Sciences Contains in Natural Science Learning for Elementary School. Journal of Research and Educational Research Evaluation, 11(1), 15-22.
Newell, S. J., Chur-Hansen, A., & Strelan, P. (2021). A construct validity analysis of the concept of psychological literacy. Australian Journal of Psychology, 73(4), 1922069.
Ni’mah, F. (2019). Research trends of scientific literacy in Indonesia: Where are we?. Jurnal Inovasi Pendidikan IPA, 5(1), 23-30.
Novaristiana, R., Rinanto, Y., & Ramli, M. (2019). Scientific literacy profile in biological science of high school students. JPBI (jurnal pendidikan biologi indonesia), 5(1), 9-16.
O’Connell, M. (2018). The power of cognitive ability in explaining educational test performance, relative to other ostensible contenders. Intelligence, 66, 122-127.
Osborne, J. (2023). Science, scientific literacy, and science education. In Handbook of research on science education (pp. 785-816). Routledge.
Ramos, J. L. S., Dolipas, B. B., & Villamor, B. B. (2013). Higher order thinking skills and academic performance in physics of college students: A regression analysis. International Journal of Innovative Interdisciplinary Research, 4(1), 48-60.
Ravand, H., & Firoozi, T. (2016). Examining construct validity of the master’s UEE using the Rasch model and the six aspects of the Messick’s framework. International Journal of Language Testing, 6(1), 1-18.
Romine, W. L., Sadler, T. D., & Kinslow, A. T. (2017). Assessment of scientific literacy: Development and validation of the Quantitative Assessment of Socio‐Scientific Reasoning (QuASSR). Journal of Research in Science Teaching, 54(2), 274-295.
Rönnebeck, S., Bernholt, S., & Ropohl, M. (2016). Searching for a common ground–A literature review of empirical research on scientific inquiry activities. Studies in Science Education, 52(2), 161-197.
Rosana, D., Widodo, E., Setianingsih, W., & Setyawarno, D. (2020). Developing assessment instruments of PISA model to measure students’ problem-solving skills and scientific literacy in junior high schools. Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education), 8(2), 292-305.
Rusilowati, A., Kurniawati, L., Nugroho, S. E., & Widiyatmoko, A. (2016). Developing an Instrument of Scientific Literacy Assessment on the Cycle Theme. International Journal of Environmental and Science Education, 11(12), 5718-5727.
Rusilowati, A., Nugroho, S. E., Susilowati, E. S. M., Mustika, T., Harfiyani, N., & Prabowo, H. T. (2018, March). The development of scientific literacy assessment to measure student’s scientific literacy skills in energy theme. In Journal of Physics: Conference Series (Vol. 983, No. 1, p. 012046).
Rusilowati, A., Nugroho, S. E., Susilowati, E. S. M., Mustika, T., Harfiyani, N., & Prabowo, H. T. (2018, March). The development of scientific literacy assessment to measure student’s scien-tific literacy skills in energy theme. In Journal of Physics: Conference Series (Vol. 983, No. 1, p. 012046). IOP Publishing.
Sireci, S. G., & Benítez Baena, I. (2023). Evidence for test validation: A guide for practitioners. Psico-thema.
Sireci, S. G., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema.
Sumintono, B. (2018, February). Rasch Model Measurements as Tools in Assesment for Learning. In 1st International Conference on Education Innovation (ICEI 2017). Atlantis Press
Susongko, P , Kusuma, M, Yuenyong, C., & Arfiani, Y. (2024). Developing the Nature of Science (NOS) Test Using Covid-19 Theme. Pegem Journal of Education and Instruction, 14(2), 57-66.
Susongko, P. (2016). Validation of science achievement test with the rasch model. Jurnal Pendidikan IPA Indonesia, 5(2), 268-277.
Susongko, P. U. R. W. O., Arfiani, Y., & Kusuma, M. (2021). Determination of Gender Differential Item Functioning in Tegal Students’ Scientific Literacy Skills with Integrated Science (SLi-SIS) Test Using Rasch Model. Jurnal Pendidikan IPA Indonesia, 10(2), 270-281.
Susongko, P., Kusuma, M., & Arfiani, Y. (2022, De-cember). 3-dimensional scientific literacy assessment framework for senior high school science program students. In AIP Conference Proceedings (Vol. 2600, No. 1). AIP Publishing.
Susongko, P., Kusuma, M., Arfiani, Y., Samsudin, A., & Aminudin, A. (2020). Revising of the Integrating Scientific Literacy Skills Scale (ISLS) with Rasch Model Analysis. Journal for the Education of Gifted Young Scientists, 8(4), 1583-1602.
Susongko, P., Widiatmo, H., Kusuma, M., & Afiani, Y. (2019). Development of Integrated Science-Based Science Literacy Skills Instruments using the Rasch Model. Unnes Science Education Journal, 8(3).277-292
Suzuri-Hernandez, L. J. (2010). Exploring school students’ views of the nature of science (Doctoral dissertation, University of York).
Veiga, F. H. (2016). Assessing student Engagement in School: Development and validation of a four-dimensional scale. Procedia-Social and Behavioral Sciences, 217, 813-819.
Wahyuni, S., Indrawati, I., Sudarti, S., & Suana, W. (2017). Developing science process skills and problem solving abilities based on outdoor learning in junior high school. Jurnal Pendidi-kan IPA Indonesia, 6(1).
Wenning, C. J., & Vieyra, R. E. (2020). Teaching High School Physics: The Nature of Physics Teaching. USA: AIP Publishing.
