IMPROVING ELEMENTARY SCHOOL STUDENTS’ CRITICAL THINKING SKILL IN SCIENCE THROUGH HOTS-BASED SCIENCE QUESTIONS: A QUASI-EXPERIMENTAL STUDY

There have been many efforts to improve elementary school students’ critical thinking skills in science through various learning methods. However, only a few research results show efforts to improve critical thinking skills through HOTS-based science questions for elementary school students. Therefore, the purpose of this study is to test the effectiveness of student habituation with HOTS-based science questions in improving elementary school students’ critical thinking skills in science. Quasi-experimental methods were employed in this research with a nonequivalent control group design involving a treatment class and a control class. The treatment class gets treatment in HOTS-based science questions habituation during the learning process, while the control class in conventional approaches. A total of 60 students is from one of the elementary schools in Surakarta, Indonesia. To take data related to students’ critical thinking skills, they were given pretest and posttest where each test used HOTS-based science questions in an essay. The obtained data from the tests were then analyzed using descriptive and inferential statistical techniques. This study showed that the average of critical thinking skills in science of students in the experimental class was higher than the control class, with a positive mean difference of 0.4226. Based on these results, it is recommended that the results of this study can provide an overview to educational practitioners at the elementary school level and researchers in the field of science education related to efforts to improve elementary school students’ critical thinking skills in science through the habituation of HOTS-based science questions. © 2021 Science Education Study Program FMIPA UNNES Semarang

INTRODUCTION PISA (Programme for International Student Assessment) in 2018 released their latest report data related to literacy levels for each member country, one of which is Indonesia. Based on the report, Indonesian students' literacy levels ranked 74th out of 79 member countries (OECD, 2018). The ranking is a concern for Indonesia because a nation's literacy level affects how strong the people's competitiveness in the era of global competition (Nair et al., 2012;Fitriansyah et al., 2020). Literacy skills can determine a nation's competitiveness in science literacy (Amaringga et al., 2021). Science literacy is a skill to use personal knowledge of science to solve problems (Feinstein, 2011;Zulfiani et al., 2020). In line with the rapid development of technology globally, science literacy becomes vital to be mastered by the Indonesian generation. Unfortunately, as in the PISA report data in 2018, Indonesian students' science literacy level is still deficient. Indonesia is ranked 70th, lags far behind neighboring countries such as Malaysia, Singapore, and Thailand. The case should be a particular concern to improve the quality of science education in Indonesia. One of the efforts that can be done is to improve students' critical thinking skills in science.
Critical thinking is a skill to think about an object, content, or problem that the thinker improves his thinking quality by skillfully taking charge of the structures inherent in thinking and imposing intellectual standards upon them (Elder & Paul, 2010). In comparison, Ennis (2011) argues that critical thinking can think logically and reflectively, deciding what to do. From both opinions, it can be concluded that critical thinking can use thinking skills by utilizing various objects and concepts to solve a problem. In other words, critical thinking skill is a tool to solve both contextual and non-contextual problems (Thayer-Bacon, 1998). A survey conducted by the American Association of Colleges and Universities (AACU) found that 75% of employers want colleges to place more emphasis on critical thinking, real-world problem solving, communication, and creativity, and 93% of these employers felt that these skills were more important than the major (Association of American Colleges and Universities, 2013). The survey is supported by another survey conducted by the American Management Association of international managers and executives showed that in 2019, critical thinking skills were ranked as the most critical skills related to growth (American Management Association, 2019). Assessing whether a person has good critical thinking skills can be seen from several aspects: analytical skill, inference skill, evaluation skill, induction skill, and deduction skill (see Table 1 for details) (Phillips et al., 2004). Efforts to improve critical thinking skills in science can be made by habituation of students with science questions based on Higher-order Thinking Skills (HOTS). Principally, Higher-order Thinking Skills (HOTS) is a skill to do something about the fact, understand it, conclude it, link it to facts in new ways, and apply it to find solutions to problems (Thomas & Thorne, 2009). Based on Bloom Taxonomy, HOTS requires mastery of thinking skills at the analysis, evaluation, and creation level (Anderson & Krathwohl, 2001). As for fulfilling the mastery, critical thinking skills are required (Misrom et al., 2020;Purnami et al., 2021). Therefore, to improve students' critical thinking skills in science, habituation is required in using HOTS-based science questions. The habituation of HOTS-based science questions should be applied as early as possible by seeing the importance of critical thinking skills in science. Therefore, the habituation of the question can be applied to students at the elementary school level (Ishartono et al., 2021b). This level is the most basic school level where the basics of knowledge are taught, including science.
Based on previous research, efforts to improve the elementary school students' critical thinking skills in science have been widely made (Forawi, 2016;Stephenson & Sadler-McKnight, 2016;Vieira & Tenreiro-Vieira, 2016). In addition, there are results of previous studies that examined the application of HOTS in improving elementary school students' critical thinking skills, as was done by Permana & Utomo (2021). They applied the HOTS concept to thematic teaching books assisted by 3D Stereoscopic Images to improve the elementary school students' critical thinking skills. However, based on those previous studies, there is scant in improving the elementary school students' critical thinking skills through HOTS-based science questions. This situation is critical to be studied and explored more deeply because improving critical thinking skills early can help students prepare themselves in the face of global competition through the science aspect. The research question is whether the habituation of HOTS-based science problems effectively improves the critical thinking skills of elementary students in science. Therefore, the purpose of this study is to test the effectiveness of habituation methods with HOTS-based science problems in improving elementary school students' critical thinking skills in science.
The hypothesis in this study is that the habituation method of HOTS-based science effectively improves elementary school students' critical thinking skills in science. The limitation of this study is only in the indicator of the students' critical thinking skills (participants). Their mental activity or condition is not taken into account.

METHODS
This study is an experimental study with a quasi-experiment model where authors compare students' critical thinking skills level in science from treatment and control class (Cook, 2015). The treatment group is a class that is given treatment in habituation with HOTS-based science questions. It is based on the statement given by Hannel & Hannel (1998) that one step of critical thinking is to perform an analysis that is also accommodated in the type of HOTS problems so that the relationship between HOTS-based science questions with critical thinking skills is the requirement of analysis step in completing HOTS-based science questions. At the same time, a control class is taught using a lecturing method where the teacher conducts the learning process by only explaining the material directly to students (Benjamin Jr, 2013). The pretest and posttest were used for each sample class to see the improvement of critical thinking skills of both classes (see Table 2 for the design). In Table 2, the O sign means that the action is implemented, while the X sign means that the action is not implemented. The first step in this study is to conduct a pretest both in the treatment class and in the control class to determine the students' initial critical thinking skills in both sample classes. The data from this stage is first analyzed using a prerequisite statistical test before testing balance level using T-Test. If it is declared balanced, both sample classes are given treatment as planned in Table 2. Furthermore, a posttest is used to obtain data on the level of students' critical thinking skills. This posttest data is also done prerequisite statistical analysis first, then conducted a T-test to see the difference in average critical thinking skill of students from both sample classes. This research was conducted at one of the public elementary schools in Surakarta, Indonesia, in the fifth grade of six parallel classes. The school was chosen because it has the same program that aims to implement HOTS-based science questions. The sampling technique of this study was the random cluster technique since there are no specific characteristics of the classes. From the six classes, selected two classes randomly. One class becomes an experiment class, and the other becomes a control class, with 30 students each. The 60 students previously have obtained relevant science materials to be a participant in this study. The condition of students from both classes has average skills that are commensurate.
Based on the research purpose, the data collection techniques in this study are tests and documentation. The test is based on HOTSbased science questions applied at the pretest and posttest. The questions in the test include materials related to gestures and human devices. The test consists of three HOTS-based essay questions in the pretest and posttest that were developed using the syntax of developing HOTS-based science questions for the elementary school level (Ishartono et al., 2021a). Two experts in science education from a state university in Indonesia assessed it. The assessment of the two experts was conducted inter-raters Cohen's Kappa reliability test with the result that the agreement level of the two experts is relatively strong. It can be continued to content validity test by using Aiken's coefficient value which each question scored above 0.8 or categorized as valid (Retnawati, 2016). In addition, these questions are also tested for reliability levels using ANATES V4 software (Ariany & Al-Ghifari, 2018), and it was obtained that the reliability value was at point 0.64 or categorized as high (Bendig, 1954). Thus, the result is that each question tested is valid and can be used for pretest and posttest. At the same time, the documentation technique is used to collect the data needed during the research process.
To see students' critical thinking skills based on pretest and posttest results, the author created an assessment card based on critical thinking aspects, as in Table 1. The assessment score is based on the semantic differential scale with category 1, "very low," and 5, "very high" (Gliner et al., 2016). The assessment was conducted by two assessors, the first and the third author. The assessment results of the two assessors have then tested the level of agreement using the reliability test inter-raters Cohen's Kappa. If the level of agreement of the two assessors is relatively high, then the data of pretest and posttest assessment results can be used for the further statistical test. Finally, assessment techniques were conducted with the average value of all aspects of students' critical thinking skills in pretest and posttest scores.
This study used two analytical techniques, descriptive analysis (the data obtained is described in-depth) and inferential techniques (used to review, estimate, and draw conclusions based on the data obtained). Several tests were used in inferential techniques, such as statistic prerequisite tests on pretest data such as homogeneity and normality tests (Scott & Usher, 1996;Suana et al., 2017). In addition, inferential techniques were used for hypothesis testing using the inde-pendent sample t-test formula on students' pretest and posttest scores (Field, 2013;Zulfiani et al., 2020). All the tests were analyzed by using SPSS 23.

RESULTS AND DISCUSSION
The first prerequisite test is done before the data is analyzed in advanced statistics: the normality and homogeneity tests (Odewumi et al., 2019). The normality test used the Shapiro-Wilk formula because samples per class are 30 students (Wilkinson & Birmingham, 2014). Based on the Shapiro-Wilk normality test results, the data in both classes are distributed normally, with each significance value above 0.05 (see Table 3). The homogeneity test using the Levene formula because the present study only tested two data groups (Gliner et al., 2016). It is obtained that the significance value is at 0.643, which means that the data of both classes are homogeneous (see Table 4). Before conducting the hypothesis test, the authors first tested the agreement level of the assessors based on the assessment results of the two designated assessors using Cohen's Kappa interrater reliability test. The assessment results are te-sted only on assessing aspects of analytical skills in the pretest stage in the control class. The results of the agreement level test of the two assessors obtained a value of 0.824 or were classified as having a strong agreement level (see Table 5). With the strong agreement between assessors, the authors used the pretest data to test the average similarity level of two samples using an independent sample t-test formula. This test uses the H 0 hypothesis for conditions where there is a significant difference between critical thinking skills in science for the control class and the experimental class (Sig. (2-tailed) < 0.05), and H 1 for conditions where there are no significant dif-ferences between the control class and the experimental class (Sig. (2-tailed) ≥ 0.05). Based on the test results using SPSS 23, it was obtained that the value of Sig (2-tailed) was at 0.296 or Sig. (2-tailed) ≥ 0.05, or in other words that there were no significant differences in critical thinking skills in science between the control class and the experimental class (see Table 6). It also shows in Table 7 that the average difference of critical thinking skills in the science of both classes is not significant, which is closely the same. The treatment of both classes can be carried out before the posttest. The results were analyzed to test the effectiveness of both classes. After it was stated that both classes were equal, and the learning process and posttest were completed under the research process, the same two assessors assessed students' posttest scores from both classes. The assessment of the two assessors has then analyzed its normality by using the Shapiro-Wilk test and its homogeneity by using the Levene test. Based on the normality test, it was obtained that the significance value of this test was 0.528, which means that the posttest data was normally distributed (see Table 8). Based on the homogeneity test, it was obtained that the significance value of the test was 0.178 or can be categorized as homogenous (see Table 9). Therefore, the posttest data can be used for hypothesis tests using an independent sample t-test. An independent sample t-test was used to see if there was a difference in critical thinking skills in science between the control class and the experimental class. The used hypothesis was the same as what was used to analyze the pretest score that was H 0 hypothesis for conditions where there is a significant difference between critical thinking skills in science for the control class and the experimental class (Sig. (2-tailed) < 0.05), and H 1 for conditions where there are no significant differences between the control class and the experimental class (Sig. (2-tailed) ≥ 0.05). Table 10 shows the value of Sig. (2-Tailed) of the t-test results in the posttest data is below 0.05, or H 0 is rejected. It can be said that there is a significant mean difference between the control and the experimental class. Therefore, descriptive tests were conducted to compare which of the two classes had a larger average. Based on the results of descriptive statistics tests on posttest data from both classes, it was obtained that the average critical thinking skill in science in the control class was 3.5467, while in the experimental class was 3.9733 (see Table 11). This result indicates that the average level of critical thinking skill in the science of the experimental class is better than the control class. For more details, Table 12 shows that the improvement of critical thinking skills in every aspect of the experimental class is more significant than in the control class. It means that the habituation method of elementary school students with HOTS-based science questions is considered to improve their critical thinking skills in science. The impact of globalization forces all nations to improve their competitive capabilities, one of which is the skill to compete in science. Mastery of science can help a nation create technologies to facilitate the nation's life further. Indeed, the mastery of science must be followed by critical thinking skills so that mastery of science can be a solution for the nation's life. Seeing the urgency of critical thinking skills in science, it takes an effort to improve them at the elementary school level as early as possible. One of the efforts is to familiarize elementary school students with HOTS-based science questions. As previously stated, no research has examined the effectiveness of this way in improving students' critical thinking skills in science. Therefore, this study examined the effectiveness of this method by using quasi-experimental methods by comparing the improvement of students' critical thinking skills in experimental and control classes.
Before implementing HOTS-based science question habituation for the students, the authors previously provided a piece of training for the teachers on developing HOTS-based science questions appropriate for elementary school stu-dents following the syntax of hots-based science question development (Ishartono et al., 2021a). The training was carried out two days earlier with external in the form of HOTS-based science question questions that they will later use while implementing the method. After that, the authors and teachers conducted a focused group discussion (FGD) to implement HOTS-based science question habituation to students.
After the training and FGD are completed, the next teacher applies the HOTS-based science question habituation process to students based on the teacher's problems. The scenario implemented is teachers giving material first based on a case that the teacher will use to formulate concepts. After students are considered to understand the concept, the teacher gives two questions. The first question is a routine problem related to the concept, and the second question is a HOTS-based science question related to the concept. The proportion of learning time increases the students' experience with HOTS-based science questions and discussions with teachers.
The authors began data retrieval by pretesting HOTS-based science questions to students in experimental and control classes. The students' answers have assessed critical thinking skill level using the aspects in Table 1. The assessment results are then tested using statistical prerequisite tests, which are normality tests and homogeneity tests. For the normality test results using the Shapiro-Wilk test, as stated in Table 3, the results were obtained that students' critical thinking skill in both classes was distributed normally. Next is to test homogeneity using the Levene formula. Based on Table 4, it is obtained that both classes are homogeneous. Because both prerequisite tests are met, testing pretest data is continued with a t-test to see the balance of students' skills in experimental and control classes. Table 6 shows that students' critical thinking skills in science in both classes are no different. Therefore, both classes are balanced and can be used as samples for treatment according to the research procedure described earlier.
The data retrieval process was conducted in four weeks, where authors collaborated with teachers at one of the elementary schools in Surakarta, Indonesia, to familiarize students with HOTS-based science questions in experimental classes. While in the control class, the learning process is carried out conventionally, namely direct learning. Due to the COVID-19 pandemic and the limitations of online facilities owned by some students, the learning process in experimental classrooms and control classes is blended, i.e., some students carry out online and others come to school. In the fourth meeting, the authors give the students in both classes a posttest in the form of HOTS-based science questions.
The posttest data is then used to test its normality and homogeneity before entering the hypothesis test using a t-test. As for the normality test results, as in Table 8, the posttest values of both classes are the same, which means the data was distributed normally. Based on Table  9, it appears that both classes are homogeneous. Because the posttest data is distributed normally and homogeneously, the author continues the hypothesis test using the t-test formula. Based on Table 10, it was found that there is an average difference in elementary school students' critical thinking skills level in control and experimental classes. The mean score for the experimental class (0.4226) is higher than the mean score of the control class (see Table 11). It means that habituation of HOTS-based science questions effectively improves critical thinking skills in science for elementary school students.
By observing the increase in the average of students' critical thinking skill in science in both sample classes, there is a significant difference that can also indicate that students' critical thinking skills in science in experimental classes is higher than in the control class (see Table 7 and Table 11). The details can also be seen in Table  12, where the experimental class has a higher significance difference between pretest and posttest in each aspect than the control class. It shows that the habituation method of elementary school students with HOTS-based science questions effectively improves their critical thinking skills in science.
The results of this study are in line with Widana et al. (2018), which shows that the use of HOTS-based questions can help teachers improve students' critical thinking skills. If drawn to improving critical thinking skills in science, this study also supports the research results from Saputri et al. (2019), which stated that HOTS-based questions also improve students' thinking skills in learning science with different levels of research subjects. The improvement of mastery of critical thinking also directly affects analytical skills, which can help students map problems and information needed as capital to solve a problem (Chijioke & Offiah, 2013;Cullen et al., 2018). This statement aligns with Taleb & Chadwick (2016), where critical thinking skills can improve analyzing skills. In addition to improving students' analytical skills, improving critical thinking skills can also improve students' skills to conclude (Cañongo et al., 2020). Inferential skills are very beneficial for students, especially when they learn to make decisions related to strategies to solve a problem (Soto et al., 2019).
Based on the discussion, it can be seen that mastery of critical thinking skills can provide significant benefits for students, especially in the context of problem-solving related to science. It is considered very important because critical thinking skills are 21st-century skills that will help students compete globally (Aslan, 2015). Therefore, the results of this study are significant as a reference for educators in science about how to develop students' critical thinking skills through HOTS-based science questions. Besides, it is expected that mastery of critical thinking skills in science from an early age can be a significant starting capital for a nation to improve its welfare. Indeed, this study still requires further deepening due to time constraints and the number of subjects involved. Therefore, the expansion of the subject and deepening of the analysis, such as comparing HOTS-based science questions with other types of questions, can be done in subsequent research.