Evaluating STEM-based Reform Teaching Observation Protocol to Enhance Students’ Communication Skills in the Context of Sustainable Development Goals in Science Learning
DOI:
https://doi.org/10.15294/c97mez74Keywords:
Communication Skills, STEM, RTOP, Science Learning, SDGsAbstract
STEM-based teaching can be challenging as it requires the integration of science, technology, engineering, and mathematics in one lesson. Selecting an engaging topic involving students’ experiences with current issues can be helpful in this regard. The Sustainable Development Goals (SDGs) are a relevant and impactful topic that may facilitate students’ communication skills. This research aimed to evaluate the Reform Teaching Observation Protocol (RTOP) using a STEM approach to address SDGs and to improve students’ communication skills. This research employed quantitative approach to measure an instrument for teaching STEM through the lens of SDG-related issues in the science learning. The process included creating the items, reviewing the RTOP, and testing these items with participants who are the representatives of the intended users. Exploratory Factor Analysis (EFA) was used to measure the structural correlations among items and topics and assess their internal reliability. The results identified five factors in the RTOP evaluation: developing scientific topics (8 items), skills in conveying specific examples (5 items), use of scientific language terms (2 items), representative forms of scientific evidence (2 items), and involving the STEM approach (2 items). The reliability of the RTOP, as indicated by Cronbach’s Alpha, was 0.818. Based on these findings, the research concludes that the RTOP is both valid and reliable, and can be implemented in the classroom to teach STEM and SDGs, thereby improving students’ communication skills.
References
AlAli, R., Alsoud, K., & Athamneh, F. (2023). To-wards a Sustainable Future: Evaluating the Ability of STEM-Based Teaching in Achieving Sustainable Development Goals in Learning. Sustainability, 15(16), 12542.
Albareda-Tiana, S., Fernandez-Borsot, G., Berbegal-Mirabent, J., Regadera González, E., Mas-Machuca, M., Graell, M., Manresa, A., Fernández-Morilla, M., Fuertes-Camacho, M. T., Gutiérrez-Sierra, A., & Guardiola, J. M. (2024). Enhancing Curricular Integration Of The Sdgs: Fostering Active Methodologies Through Cross-Departmental Collaboration In A Spanish University. International Journal of Sustainability in Higher Education, 25(5), 1024–1047.
Bautista, C., Alfuraiji, N., Drangowska-Way, A., Gangwani, K., de Flamingh, A., & Bourne, P. E. (2022). Ten Simple Rules For Improv-ing Communication Among Scientists. PLoS Computational Biology, 18(6), 1- 14.
Bazan, C. (2023). Factor Analysis of the University Environment and Support System: A Bayesian Approach. SAGE Open, 13(4), 1- 19.
Beardsworth, S. J. (2020). Building Knowledge Bridges through Effective Science Communication. Chemistry - A European Journal, 26(8), 1698–1702.
Becerra, B., Núñez, P., Vergara, C., Santibáñez, D., Krüger, D., & Cofré, H. (2023). Developing an Instrument to Assess Pedagogical Content Knowledge for Evolution. Research in Science Education, 53(2), 213–229.
Benson, J., & Clark, F. (1982). A Guide for Instrument Development and Validation. The American Journal of Occupational Therapy, 36(12), 789–800. https://doi.org/10.5014/ajot.36.12.789.
Carragher, N., Teesson, M., Sunderland, M., Newton, N. C., Krueger, R. F., Conrod, P. J., Barrett, E. L., Champion, K. E., Nair, N. K., & Slade, T. (2016). The Structure of Adolescent Psychopa-thology: A Symptom-Level Analysis. Psycho-logical Medicine, 46(5), 981–994.
Chaigneau, T., Coulthard, S., Brown, K., Daw, T. M., & Schulte-Herbrüggen, B. (2019). Incorporating Basic Needs to Reconcile Poverty and Eco-system Services. Conservation Biology, 33(3), 655–664.
Cian, H., & Cook, M. (2020). Secondary Science Student Teachers’ Use of Verbal Discourse to Communicate Scientific Ideas in Their Field Placement Classrooms. Research in Science Education, 50(4), 1389–1416.
Clark, S., Maclachlan, M., Marshall, K., Morahan, N., Carroll, C., Hand, K., Boyle, N., & O’Sullivan, K. (2022). Including Digital Connection in the United Nations Sustainable Development Goals: A Systems Thinking Approach for Achieving the SDGs. Sustainability (Switzer-land), 14(3), 1883.
Contera, S. (2021). Communication is Central to The Mission Of Science. In Nature Reviews Materials, 6(5), 377–378.
Diemer, M. A., Rapa, L. J., Park, C. J., & Perry, J. C. (2017). Development and Validation of the Critical Consciousness Scale. Youth and Society, 49(4), 461–483. https://doi.org/10.1177/0044118X14538289.
Elmer, S. (2024). Science Communication Training for Graduate Exercise Physiology Students. Physiology, 39(S1), 2603.
Fatihah, A. Al, Yennita, Y., & Futra, D. (2022). Stu-dents’ Written Communication Skills in Science Learning. Journal of Innovation in Edu-cational and Cultural Research, 3(4), 564–572.
Finch, W. H. (2023). A Comparison of Methods for Determining the Number of Factors to Retain with Exploratory Factor Analysis of Dichotomous Data. Psych, 5(3), 1004–1018.
Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40(7), 3510–3521.
Großmann, L., & Krüger, D. (2024). Assessing the quality of science teachers’ lesson plans: Evaluation and application of a novel instrument. Science Education, 108(1), 153–189.
Guo, C., Huang, Y., & Chen, X. (2024). Research on Integration of the Sustainable Development Goals and Teaching Practices in a Future Teacher Science Education Course. Sustain-ability, 16(12), 4982.
Haslbeck, J. M. B., & van Bork, R. (2022). Estimating the Number of Factors in Exploratory Factor Analysis via Out-of-Sample Prediction Errors. Psychological Methods, 29(1), 48–64.
Hudson, P., English, L. D., Dawes, L., King, D., & Baker, S. (2015). Exploring links between pedagogical knowledge practices and student out-comes in STEM education for primary schools. Australian Journal of Teacher Education, 40(6), 134–151.
Islam, N., & Jirattikorn, A. (2024). Breaking gender barriers in STEM education for achieving the SDG of quality education in Bangladesh. Development in Practice, 34(1), 129–135.
Izadi-Avanji, F. S., Zamani, N., Azizi-Fini, I., & Mohammadnejad, E. (2024). Development and psychometric testing of the Aesthetics of Everyday Life Scale in Aging (AELSA). BMC Geriatrics, 24(1). 259-268
.Jongwon, P., Young-Shin, P., Youngmin, K., Jong-seok, P., & Jin-Su, J. (2014). The Development Of The Korean Teaching Observation Protocol (KTOP) For Improving Science Teaching And Learning. Journal of Baltic Science Education , 13(2), 259–275.
Khasanah, S. R. U., Zulirfan, Z., & Syahril, S. (2023). Student Perceptions of Collaboration and Scientific Communication Skills in Physics. Jurnal Paedagogy, 10(2), 468 – 475.
Kulgemeyer, C. (2018). Impact of Secondary Students’ Content Knowledge on Their Communication Skills in Science. International Journal of Science and Mathematics Education, 16(1), 89–108.
Lin, K. Y., Wu, Y. T., Hsu, Y. T., & Williams, P. J. (2021). Effects of infusing the engineering de-sign process into STEM project-based learning to develop preservice technology teachers’ engineering design thinking. International Journal of STEM Education, 8(1). 1 – 15.
Martín-Sánchez, A., González-Gómez, D., & Jeong, J. S. (2022). Service Learning as an Education for Sustainable Development (ESD) Teaching Strategy: Design, Implementation, and Evaluation in a STEM University Course. Sustainability (Switzerland), 14(12), 6965.
Mataniari, R., Willison, J., Hasibuan, M. H. E., Sulistiyo, U., & Dewi, F. (2020). Portraying students’ critical thinking skills through research skill development (RSD) framework: A case of a biology course in an Indonesian University. Journal of Turkish Science Education, 17(2), 302–314.
Mercer-Mapstone, L. D., & Matthews, K. E. (2017). Student perceptions of communication skills in undergraduate science at an Australian re-search-intensive university. Assessment & Evaluation in Higher Education, 42(1), 98–114.
Mercer-Mapstone, L., & Kuchel, L. (2015). Teaching Scientists to Communicate: Evidence-based assessment for undergraduate science education. International Journal of Science Education, 37(10), 1613–1638.
Moore, T. J., Stohlmann, M. S., Wang, H. H., Tank, K. M., Glancy, A. W., & Roehrig, G. H. (2014). Implementation and integration of engineering in K-12 STEM education. In Engineering in pre-college settings: Synthesizing research, policy, and practices. Purdue University Press.
Nagdi, M., Leammukda, F., & Roehrig, G. (2018). Developing identities of STEM teachers at emerging STEM schools. International Journal of STEM Education, 5(1), 1–13.
NGSS. (2013). Next generation science standards: For states, by states. National Academies Press.
Ong, Y. S., Koh, J., Tan, A.-L., & Ng, Y. S. (2024). Developing an Integrated STEM Classroom Observation Protocol Using the Productive Disciplinary Engagement Framework. Research in Science Education, 54(1), 101–118.
Pérez-Rivas, F. J., Jiménez-González, J., Bayón Cabeza, M., Belmonte Cortés, S., de Diego Díaz-Plaza, M., Domínguez-Bidagor, J., Gar-cía-García, D., Gómez Puente, J., & Gómez-Gascón, T. (2023). Design and Content Validation using Expert Opinions of an Instrument Assessing the Lifestyle of Adults: The ‘PONTE A 100’ Questionnaire. Healthcare (Switzer-land), 11(14), 2038.
Putra, P. D. A., Ahmad, N., Budiarso, A. S., Indrawati, & Lestari, E. A. (2023). Development Of Argumentation Tools Based On The Engineering Design Process To Improve Students’ Argumentation Skills. The New Educational Re-view, 67(1). 114-125.
Putra, P. D. A., & Kumano, Y. (2018). Energy Learning Progression and STEM conceptualization among pre-service science teachers in Japan and Indonesia. New Educational Review, 53(3), 153–162.
Putra, P. D. A., Sulaeman, N. F., Lesmono, A. D., Kumano, Y., & Fadzil, H. B. M. (2023). Gender roles in engineering design process activity: A small group exploration through collaborative argumentation. Kasetsart Journal of Social Sci-ences, 44(1), 251-256.
Putra, P. D. A., Sulaeman, N. F., Supeno, & Wahyuni, S. (2023). Exploring Students’ Critical Thinking Skills Using the Engineering Design Process in a Physics Classroom. Asia-Pacific Education Researcher, 32(1), 141–149.
Putra, P. D. A., Yusmar, F., Kasah, I. J., Handayani, R. D., & Fadzil, H. M. (2024). Evidence-Based Reasoning: Evaluating Daily Experiences in The Engineering Design Classroom for Middle School Students. Jurnal Pendidikan IPA Indonesia, 13(1), 29–39.
Ramos-Gavilán, A. B., Rodríguez-Esteban, M. A., Frechilla-Alonso, M. A., Raposeiras, A. C., Movilla-Quesada, D., & González-Rogado, A. B. (2024). Raising awareness of the important role of engineering in sustainable development. Heliyon, 10(1), 1-14.
Reis, C. A. C. L., Nascimento do Carmo, C., Santos Nascimento Filho, A., & Lima de Souza, A. L. (2024). Evidence of validation in a strategy development model structure in public health schools. F1000Research, 13(179), 1 – 13.
Roehrig, G., Dare, E., Wieselmann, J., & Ring-Wha-len, E. A. (2021). Understanding coherence and integration in integrated STEM curricu-lum. 8(2), 1–39.
Rotger, N., Roig-Sanz, D., & Puxan-Oliva, M. (2019). Introduction: Towards A Cross-Disciplinary History of The Global In The Humanities And The Social Sciences. In Journal of Global His-tory.14(3), 325–334.
Scharfenberg, F.-J., & Bogner, F. X. (2019). A Role-Play-Based Tutor Training in Preservice Teacher Education for Developing Procedural Pedagogical Content Knowledge by Optimizing Tutor–Student Interactions in the Context of an Outreach Lab. Journal of Science Teacher Education, 30(5), 461–482.
Shivni, R., Cline, C., Newport, M., Yuan, S., & Bergan-Roller, H. E. (2021). Establishing A Base-line of Science Communication Skills in An Undergraduate Environmental Science Course. International Journal of STEM Education, 8(1), 1- 15.
Siregar, N., & Siregar, N. (2022). Peningkatan Ke-mampuan Komunikasi Sains Siswa SD melalui Pembelajaran Argument Driven Inquiry Pendekatan Kontekstual. Dikdas Mattapa: Jurnal Pendidikan Dasar, 5(2), 256-268.
Sulaeman, N. F., Putra, P. D. A., Mineta, I., Hakamada, H., Takahashi, M., Ide, Y., & Kumano, Y. (2021). Exploring Student Engagement in STEM Education through the Engineering De-sign Process. Jurnal Penelitian Dan Pembelajaran IPA, 7(1), 1-16.
Thiel, M., Baeza Álvarez, J., Diaz, M., de Veer, D., Dittmann, S., Guevara-Torrejón, V., Ahumada, G. H., Honorato-Zimmer, D., Kiessling, T., Leyton Muñoz, A., López-Xalín, N., Nuñez, P., Sepúlveda, J. M., & Vásquez, N. (2023). Communication strategies in an international school citizen science program investigating marine litter. Frontiers in Environmental Science, 11(12). 1-12.
Walker, D., Patten, T., & Stephens, S. (2024). Peer observation of teaching in a post-primary context. Irish Educational Studies, 43(1), 47–60.
Walsh, Z., Böhme, J., & Wamsler, C. (2021). Towards a relational paradigm in sustainability research, practice, and education. Ambio, 50(1), 74–84.
Waltner, E. M., Rieß, W., & Mischo, C. (2019). Development and validation of an instrument for measuring student sustainability competencies. Sustainability (Switzerland), 11(6). 1 – 16.
Watson, J. C. (2017). Establishing evidence for inter-nal structure using exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 50(4), 232–238. Yildiz, C., & Guler Yildiz, T. (2021). Exploring the relationship between creative thinking and scientific process skills of preschool children. Thinking Skills and Creativity, 39(1), 1-12.
Yustika, D., Dwi Aristya Putra, P., Prihatin, J., & Jember, U. (2023). Analysis of Students’ Argumentation Skill in Science Learning using Engineering Design Process. Unnes Science Education Journal, 12(3), 116–121.
Zabaniotou, A. (2020). Towards gender equality in Mediterranean Engineering Schools through networking, collaborative learning, synergies and commitment to SDGs-The RMEI approach. Global Transitions, 2, 4–15.
Zwolińska, K., Lorenc, S., & Pomykała, R. (2022). Sustainable Development in Education from Students’ Perspective—Implementation of Sustainable Development in Curricula. Sustainability (Switzerland), 14(6), 1-16.
