The Role and Evaluation of ChatGPT as a Virtual Tutor in Improving Students' Creative and Critical Abilities Reviewed from Probing-Prompting Abilities
DOI:
https://doi.org/10.15294/a2g5x690Keywords:
ChatGPT, Creative, Critical, Probing-Prompting, Virtual TutorAbstract
This study reveals the potential of ChatGPT in supporting students' mathematics learning. However, the study's results also show challenges in utilising ChatGPT, such as inconsistent answer quality and potential student dependence on technology. This indicates the need for a more holistic approach to integrating ChatGPT into the learning process. This study aimed to analyse ChatGPT's ability to answer quantitative knowledge and mathematical reasoning questions, assess students' prompting abilities with ChatGPT, evaluate the use of ChatGPT, and measure the extent to which students' mathematical, creative and critical abilities have increased. This study uses an Embedded Design type of Mixed Method approach. The population in this study were all high school students who took the Tridaya Tutoring Program 2024/2025. The sampling technique used was purposive sampling, and the sample selected was all 20 students from class XII of Tridaya Tutoring High School. The research instruments used were written tests, namely multiple-choice questions and documentation studies of screenshots of ChatGPT answer results. The study results showed that the t value for measuring creative and critical abilities was 6.691 and 7.151. It was found that there was a significant difference between the test scores obtained before and after the treatment. This shows that the instructions given to students can improve their test scores. ChatGPT can help students in learning as a virtual tutor. However, it should also be noted that not all questions can be answered. Some even have different answers. This makes students dissatisfied with the results obtained. This is where students are trained in critical thinking why this can happen. In addition, the student's prompting technique for ChatGPT is also considered. There is a need for creativity in prompting so students can further explore the answers from ChatGPT. Although they can answer any question, guidance from the teacher and independent verification are needed regarding the answers given by ChatGPT so that the desired answer is by expectations.
Penelitian ini mengungkap potensi ChatGPT dalam mendukung pembelajaran matematika siswa. Namun, hasil penelitian juga menunjukkan tantangan dalam memanfaatkan ChatGPT, seperti kualitas jawaban yang tidak konsisten dan potensi ketergantungan siswa pada teknologi. Hal ini menunjukkan perlunya pendekatan yang lebih holistik untuk mengintegrasikan ChatGPT ke dalam proses pembelajaran. Penelitian ini bertujuan untuk menganalisis kemampuan ChatGPT dalam menjawab pertanyaan pengetahuan kuantitatif dan penalaran matematika, menilai kemampuan prompting siswa dengan ChatGPT, mengevaluasi penggunaan ChatGPT, dan mengukur sejauh mana kemampuan matematika, kreatif, dan kritis siswa meningkat. Penelitian ini menggunakan pendekatan Mixed Method tipe Embedded Design. Populasi dalam penelitian ini adalah seluruh siswa SMA yang mengikuti Program Bimbingan Belajar Tridaya 2024/2025. Teknik pengambilan sampel yang digunakan adalah purposive sampling, dan sampel yang dipilih adalah seluruh siswa kelas XII SMA Bimbingan Belajar Tridaya yang berjumlah 20 orang. Instrumen penelitian yang digunakan adalah tes tertulis, yaitu soal pilihan ganda dan studi dokumentasi berupa tangkapan layar hasil jawaban ChatGPT. Hasil penelitian menunjukkan bahwa nilai t untuk mengukur kemampuan kreatif dan kritis adalah 6,691 dan 7,151. Ditemukan bahwa terdapat perbedaan yang signifikan antara nilai tes yang diperoleh sebelum dan sesudah perlakuan. Hal ini menunjukkan bahwa petunjuk yang diberikan kepada siswa dapat meningkatkan nilai tes mereka. ChatGPT dapat membantu siswa dalam belajar sebagai tutor virtual. Namun, perlu diperhatikan juga bahwa tidak semua soal dapat dijawab. Bahkan ada yang jawabannya berbeda-beda. Hal ini membuat siswa tidak puas dengan hasil yang diperoleh. Di sinilah siswa dilatih untuk berpikir kritis tentang mengapa hal tersebut dapat terjadi. Selain itu, teknik prompting siswa untuk ChatGPT juga diperhatikan. Perlu adanya kreativitas dalam prompting agar siswa dapat lebih mengeksplorasi jawaban dari ChatGPT. Meskipun dapat menjawab soal apa saja, diperlukan arahan dari guru dan verifikasi mandiri terkait jawaban yang diberikan ChatGPT agar jawaban yang diinginkan sesuai dengan harapan.
Downloads
References
Asad, M. M., & Ajaz, A. (2024). Impact of ChatGPT and generative AI on Lifelong Learning And Upskilling Learners In Higher Education: Unveiling The Challenges And Opportunities Globally. The International Journal of Information and Learning Technology, 41(5), 507–523. https://doi.org/10.1108/IJILT-06-2024-0103
Atlas, S. (2023). ChatGPT for Higher Education and Professional Development: A ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI Guide to Conversational AI Terms of Use.
Atlas, S. (2023). ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI. Retrieved from https://digitalcommons.uri.edu/cba_facpubs/548
Bai, Y., Liang, H., Qi, C., & Zuo, S. (2023). An Assessment of Eighth Graders’ Mathematics Higher Order Thinking Skills in the Chinese Context. Canadian Journal of Science, Mathematics and Technology Education, 23(2), 365–382. https://doi.org/10.1007/s42330-023-00279-w
Barana, A., Marchisio, M., & Roman, F. (2023). Fostering Problem Solving and Critical Thinking in Mathematics Through Generative Artificial Intelligence. In 20th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2023), 377-385
Deering, K., & Williams, J. (2020). Approaches to reviewing the literature in grounded theory: a framework. Nurse Researcher, 28(4), 9–15. https://doi.org/10.7748/nr.2020.e1752
Putri, D.A., Sayyida Hilmia, R., Almaliyah, S., Permana, S., & Studi Bimbingan dan Konseling, P. (2023). Pengaplikasian Uji T Dalam Penelitian Eksperimen. Jurnal Lebesgue: Jurnal Ilmiah Pendidikan Matematika, Matematika dan Statistika, 4(3), 1978-1987. https://doi.org/10.46306/lb.v4i3
Ding, L., Li, T., Jiang, S., & Gapud, A. (2023). Students’ perceptions of using ChatGPT in a physics class as a virtual tutor. International Journal of Educational Technology in Higher Education, 20(63), 18 pages. https://doi.org/10.1186/s41239-023-00434-1
Disnawati, H., Ndapa Deda, Y., Haning, F. O., & Pallo, M. (2022). PKM Bimbingan Menyelesaikan Soal Tes Potensi Skolastik (TPS) dalam Mempersiapkan Siswa Mengikuti UTBK-SBMPTN Community Services in Solving Scholastic Potential Test (TPS) for Preparing Students to Take UTBK-SBMPTN. Jurnal Masyarakat Mengabdi Nusantara, 1(3), 15-24.
Fahmi, M., Gunarti, A., Lestari, D. (2020). Hubungan Antara Pengelolaan Pembelajaran Dengan Tingkat Kepuasan Peserta Didik di Lembaga Bimbingan Belajar Plus Ilhami. Journal Pendidikan Untuk Semua, 04, 40–50.
Febrianingsih, F. (2022). Kemampuan Berpikir Kreatif Siswa dalam Memecahkan Masalah Matematis. Mosharafa: Jurnal Pendidikan Matematika, 11(1), 119-130.
Gartland, S. (2024). Affective content knowledge as foundation for critical mathematics pedagogy. Research in Mathematics Education, 26(1), 175–192. https://doi.org/10.1080/14794802.2022.2155693
Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), 11 pages. https://doi.org/10.30935/cedtech/13036
Indrawan, R., & Yaniawati, P. (2017). Metodologi Penelitian - Kuantitatif, Kualitatif, Campuran untuk Manajemen, Pembangunan dan Pendidikan (3rd ed.). Refika Aditama.
Jayanti, & Rahmawati. (2018). Pembelajaran Menggunakan Teknik Probing Prompting Berbantuan Edmodo Blended Learning Pada Materi Persamaan Diferensial Matematis Mahasiswa Di Universitas Pgri Palembang. Prosiding Seminar Nasional 21 Universitas PGRI Palembang, 458–465.
Jeon, J. (2024). Exploring AI chatbot affordances in the EFL classroom: young learners’ experiences and perspectives. Computer Assisted Language Learning, 37(1–2), 1–26. https://doi.org/10.1080/09588221.2021.2021241
Kasneci, E., Sessler, K., Uchemann, S. K. ¨, Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Unnemann, S. G. ¨, Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Urgen Pfeffer, J. ¨, Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., … Kasneci, G. (2023). ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learning and individual differences, 103, 102274.
Kemenristek-Dikti. (2018). Peraturan Menteri Riset, Teknologi, Dan Pendidikan Tinggi Republik Indonesia Nomor 60 Tahun 2018 Tentang Penerimaan Mahasiswa Baru Program Sarjana Pada Perguruan Tinggi Negeri. www.peraturan.go.id
Korkmaz Guler, N., Dertli, Z. G., Boran, E., & Yildiz, B. (2024). An Artificial Intelligence Application in Mathematics Education: Evaluating Chatgpt’s Academic Achievement in a Mathematics Exam. Pedagogical Research, 9(2), em0188. https://doi.org/10.29333/pr/14145
Lai, K. (2023). How Well Does ChatGPT Handle Reference Inquiries? An Analysis Based on Question Types and Question Complexities. College & Research Libraries, 84(6), 974–995.
Lee, U., Jung, H., Jeon, Y., Hwang, W., Moon, J., & Kim, H. (2024). Few-Shot is Enough: Exploring Chatgpt Prompt Engineering Method For Automatic Question Generation in English Education. Education and Information Technologies, 29(9), 11483–11515.
Lin, X. (2024). Exploring the Role of ChatGPT as a Facilitator for Motivating Self-Directed Learning Among Adult Learners. Adult Learning, 35(3), 156–166. https://doi.org/10.1177/10451595231184928
Lo, C. K. (2023). What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. In Education Sciences, 13(410), 15 pages. https://doi.org/10.3390/educsci13040410
Mika Munakata Ceire Monahan, E. K., & Vaidya, A. (2023). Non-traditional assessments to match creative instruction in undergraduate mathematics courses. International Journal of Mathematical Education in Science and Technology, 54(7), 1272–1287. https://doi.org/10.1080/0020739X.2022.2128452
Mohamed, M. Z. bin, Hidayat, R., Suhaizi, N. N. binti, Sabri, N. binti M., Mahmud, M. K. H. bin, & Baharuddin, S. N. binti. (2022). Artificial intelligence in mathematics education: A systematic literature review. International Electronic Journal of Mathematics Education, 17(3), em0694. https://doi.org/10.29333/iejme/12132
Monteleone, C., Miller, J., & Warren, E. (2023). Conceptualising critical mathematical thinking in young students. Mathematics Education Research Journal, 35(2), 339–359. https://doi.org/10.1007/s13394-023-00445-1
Montenegro-Rueda, M., Fernández-Cerero, J., Fernández-Batanero, J. M., & López-Meneses, E. (2023). Impact of the Implementation of ChatGPT in Education: A Systematic Review. In Computers (Vol. 12, Issue 8). Multidisciplinary Digital Publishing Institute (MDPI). https://doi.org/10.3390/computers12080153
Mubarika, M. P., Faiqoh, E., Susilawati, S., Raharjo, T. D., & Yaniawati, P. (2022). The Influence of Mathematical Creative Thinking Ability on Students’ Self-Regulated Learning Through a Scientific Approach. Kreano, Jurnal Matematika Kreatif-Inovatif, 13(1), 126–135.
Nazari, M., & Saadi, G. (2024). Developing effective prompts to improve communication with ChatGPT: a formula for higher education stakeholders. Discover Education, 3(45), 17 pages. https://doi.org/10.1007/s44217-024-00122-w
Newton, P., & Xiromeriti, M. (2024). ChatGPT performance on multiple choice question examinations in higher education. A pragmatic scoping review. Assessment and Evaluation in Higher Education, 49(6), 781–798. https://doi.org/10.1080/02602938.2023.2299059
Niloy, A. C., Akter, S., Sultana, N., Sultana, J., & Rahman, S. I. U. (2024). Is Chatgpt a menace to creative writing ability? An experiment. Journal of Computer Assisted Learning, 40(2), 919–930. https://doi.org/https://doi.org/10.1111/jcal.12929
Oktaria, K., Agustina, R., Aliyah, J., Sirodj, R. A., & Afgani, M. W. (2023). Grounded Theory. Journal Pendidikan Sains Dan Komputer, 3(1), 40–49. https://doi.org/10.47709/jpsk.v3i01.1957
Punar Özçelik, N., & Yangın Ekşi, G. (2024). Cultivating writing skills: the role of ChatGPT as a learning assistant—a case study. Smart Learning Environments, 11(1)10, 18 pages. https://doi.org/10.1186/s40561-024-00296-8
Rahman, M. M., & Watanobe, Y. (2023). ChatGPT for Education and Research: Opportunities, Threats, and Strategies. Applied Sciences, 13(9), 5783. https://doi.org/10.3390/app13095783
Ruffinelli, A., Álvarez Valdés, C., & Salas Aguayo, M. (2023). Conditions for generative reflection in practicum tutorials: the representations of tutors and preservice teachers. European Journal of Teacher Education, 46(3), 498–515. https://doi.org/10.1080/02619768.2021.1949708
Saifuddin, A. (2020). Apakah Desain Eksperimen Satu Kelompok Layak Digunakan? Jounal Kajian Keislaman Multi-Perspektif, 1(1), 1–22.
Sapkota, B., & Bondurant, L. (2024). Assessing Concepts, Procedures, and Cognitive Demand of ChatGPT-generated Mathematical Tasks. International Journal of Technology in Education, 7(2), 218–238. https://doi.org/10.46328/ijte.677
Saputra, H. (2020). Kemampuan Berpikir Kritis. Perpustakaan IAI Agus Salim, 1–7.
Sawangwan, S. (2024). ChatGPT vs Teacher Roles in Developing EFL Writing. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 14(1), 1-21. https://doi.org/10.4018/IJCALLT.361235
Setiawan, A., & Luthfiyani, U. K. (2023). Penggunaan ChatGPT Untuk Pendidikan di Era Education 4.0: Usulan Inovasi Meningkatkan Keterampilan Menulis. JURNAL PETISI (Pendidikan Teknologi Informasi), 4(1), 49-58.
Silva de Souza, R. Jr., & Borges, E. M. (2023). Teaching Descriptive Statistics and Hypothesis Tests Measuring Water Density. Journal of Chemical Education, 100(11), 4438–4448. https://doi.org/10.1021/acs.jchemed.3c00402
Usman, M., Adys, H. P., Rosmaladewi, R., & Asnur, M. N. A. (2023). The Application of Probing Prompting Learning Models in Mastering Foreign Language Vocabulary. International Journal of Language Education, 7(4), 702–710. https://doi.org/10.26858/ijole.v7i4.58366
Vong, S. A., & Kaewurai, W. (2017). Instructional model development to enhance critical thinking and critical thinking teaching ability of trainee students at regional teaching training center in Takeo province, Cambodia. Kasetsart Journal of Social Sciences, 38(1), 88–95. https://doi.org/10.1016/J.KJSS.2016.05.002
Widodo, W., Siswanto, H., & Lestari, G. D. (2022). Peran Tutor dalam Pembelajaran Virtual pada Pendidikan Kesetaraan Paket C di SKB Cerme Gresik. Aksara: Jurnal Ilmu Pendidikan Nonformal, 8(3), 2207. https://doi.org/10.37905/aksara.8.3.2207-2222.2022
Wiryanto. (2020). Proses pembelajaran matematika di sekolah dasar di tengah pandemi covid-19. Jurnal Review Pendidikan Dasar: Jurnal Kajian Pendidikan Dan Hasil Penelitian, 6(2), 125-132.
Yaniawati, P., Kariadinata, R., Sari, N. M., Pramiarsih, E. E., & Mariani, M. (2020). Integration of e-learning for mathematics on resource-based learning: Increasing mathematical creative thinking and self-confidence. International Journal of Emerging Technologies in Learning, 15(6), 60–78. https://doi.org/10.3991/ijet.v15i06.11915
Yaniawati, P., Supianti, I. I., Fisher, D., & Sa’adah, N. (2021). Development and effectiveness of mobile learning teaching materials to increase students’ creative thinking skills. Journal of Physics: Conference Series, 1918(4). https://doi.org/10.1088/1742-6596/1918/4/042081
Yaniawati, R. P. (2013). E-Learning to Improve Higher Order Thinking Skills (HOTS) of Students. Journal of Education and Learning (EduLearn), 7(2), 109-120.
Zein, S., & Gunawan, G. (2022). Prediksi Hasil FIFA World Cup Qatar 2022 Menggunakan Machine Learning dengan Python. Jurnal Riset Matematika, 2(2), 153–162. https://doi.org/10.29313/jrm.v2i2.1382
Downloads
Additional Files
Published
Article ID
14343Data Availability Statement
The article and its attachments contain the relevant data for review.
Issue
Section
Categories
License
Copyright (c) 2024 Kreano, Jurnal Matematika Kreatif-Inovatif
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
