The Implementation of High Order Thinking Skills (HOTS) Assessment to Evaluate the Students’ Reading Comprehension Achievement

____________________ Abstract ___________________________________________________________________ HOTS promote the idea of assessing students’ critical thinking, creativity , and problem-solving. However, many teachers claimed to have difficulties in designing questions and implementing them in teaching and learning process. The present study aimed to explain how teachers perceive, implement, and assess HOTS assessment in teaching reading comprehension. It employed an exploratory sequential design. Using the purposive sampling technique, two English teachers from SMA N 1 Pati were involved. The data were gathered through semi-structured interviews, document analysis, classroom observations, and tests. This study reveals that the teachers had a good perception of HOTS assessment in terms of concept and implementation. They implemented HOTS assessment using appropriate strategies and methods in teaching reading comprehension. In addition, the teachers designed the proper assessment using HOTS assessment in assessing reading comprehension. The students’ achievement also had achieved the required score in reading comprehension. All the results aligned with teachers’ perceptions, implementations, assessme nts, and students’ achievement. Although the teachers had exemplary performance, they still met difficulties due to

INTRODUCTION 2013 Curriculum has been implemented in many schools since 2013 in Indonesia. This curriculum focuses on achieving students' competencies and character-building (Ekawati, 2017). However, several studies revealed that there are still many problems faced by teachers and institutions in implementing the 2013 curriculum. The issues found were low teacher quality, inadequate infrastructure, and an unstandardized curriculum (Marijan, 2021, as cited in Anugrahwati, 2015). One of those that needs more improvement is the low quality of teachers in implementing 2013 Curriculum in their teaching-learning process and assessment.
The 2013 Curriculum demands teachers to implement High Order Thinking Skills in the teaching-learning process and assess students' achievement. Higher Order Thinking (HOT) refers to thinking at a higher level than remembering facts or dealing with someone. (Nur & Melati, 2019). This skill examines students' critical thinking in the learning process and assessment. By assessing students with the HOTS Assessment, teachers examine cognitive skills through several questions. This skill is activated when students are faced with an unusual problem, uncertain question, or question that raises a question. Therefore, teachers need to implement this skill in the teaching and learning process to meet the requirements of this curriculum.
Mastering reading skill is essential. Ekadini et al. (2018) cited in Anisah et al. (2019) stated that mastering reading skills is a fundamental factor in gaining success in the academic field. Although reading is an essential ability to master, Indonesian students still underestimate this skill. The tendency to read is still low for Indonesian students and they still have a lack of reading literacy. According to the PISA (Program for International Student Assessment), which focuses on the three basics of literacy, Indonesian students have inferior reading, mathematics, and science skills compared to other Southeast Asia. Even though they are lack reading literacy, they still hesitate to improve their reading skill; whereas, reading skill is the most required skill in doing English test. They will get confused and take a lot of time doing some exercises related to reading comprehension because they do not know the appropriate technique in reading skills. Moreover, most of them have difficulty understanding the text due to new vocabularies and questions that require high order thinking skills. As a result, many of them get a terrible score in English subjects, especially those with no interest in English.
Based on those problems, the teacher plays a significant role. This may happen when the teacher does not motivate the students and engage them in mastering reading comprehension while teaching-learning activities. Reading comprehension involves identifying the text and the process of understanding what the printed text is trying to say (Ancheta, 2018). Besides, it may also happen when the teacher does not teach the proper way in reading techniques. Those aspects can contribute to the unsuccessful achievement of students because they are not able to master the reading skills in which the primary skills for answering English tests. Therefore, the teacher should teach adequately and encourage them to have good reading literacy and achieve the requirement of 4C (Communication, Collaboration, Creativity, and Critical thinking) in learning activities.
After properly teaching reading comprehension, the teacher also should assess their understanding or knowledge to know whether the students master it well or not. Assessment system involves significant objectives of education (Rohmatul et al. 2020). There are lots of requirements that have to be fulfilled by the teacher for assessing reading comprehension. One of them is an appropriate question with the core competence in the syllabus. The relevant question proves teacher's professionalism because it influences the result of students' achievement.
Besides, one technique to assess students' reading comprehension is applyin the High Order Thinking Skill Assessment. Assessment is a process that measures HOTS's three highest levels in Bloom's Taxonomy for analyzing, evaluating, and creating (Swartz & McGuinness, 2014, as cited in Wayan, 2017. The teacher will examine the cognitive skill by assessing students using the HOTS assessment, through some questions. Moreover, some reading questions require students to think critically to understand and answer the questions because most English test types examine reading comprehension with high-level questions. Therefore, teachers have to explore this ability either in learning activities or assessing this problem.
However, teachers need accurate questions based on the words of the cognitive process to assess higher levels of thinking skills, especially reading comprehension. The teacher also needs good competencies in making the question based on the learning objective that the teacher wants to achieve. Many teachers still face difficulties in making a moral question to assess higher thinking skills in reading comprehension. In a preobservation stage, we found this problem when the teachers joined the Seminar about evaluating higher thinking skills in the English Teachers Association Meeting. They still used lower-order thinking skills to assess reading comprehension, which was against the demand of the 2013 Curriculum. Another problem came when the teachers created high-order thinking skill questions, but they did not achieve the learning objective.
Based on the explanation above, in this research, we attempt to explain English teachers' perceptions and actual implementations of High Order Thinking Skill Assessment to evaluate students' reading comprehension achievement in SMA N 1 Pati. From the research findings, we hope that this research provides adequate and meaningful information and recommendation for senior high school English teachers on designing and implementing a good and proper assessment for students' reading comprehension.

METHOD
This study assumed that teachers have a good understanding of the concept of high-order thinking skills for reading comprehension and understand how to implement it properly. It also assumes that teachers might get confused in designing assessments for reading comprehension using higher thinking skills. Moreover, this study assumes that students score low in reading comprehension using higher thinking skills.
The subjects of the study were two English teachers at SMA N 1 Pati in the academic year 2020/2021. They are active teachers who teach English lessons in every grade. They have proper knowledge and ability, and they have good management in the classroom and discipline as well. Meanwhile, the objects of the study were teachers' perceptions and implementations of High Order Thinking Skill Assessment to assess students' reading comprehension achievement at Senior High School.
In collecting the data, we adopted the steps suggested by Creswell (2014). We gathered qualitative data first and built on its analysis using quantitative data. The qualitative data were obtained through interviews, observation sheets, and document analysis. In addition, the quantitative data were taken by the test.
In data analysis, we adopted the steps suggested by Creswell (2012). There are four steps for analyzing and interpreting exploratory sequential data. They are: find an instrument, develop an instrument, form categorical data, and use extreme qualitative cases.

Teachers' Perception
The interview results revealed that the teachers positively perceived the implementation of HOTS assessment in teaching and assessing reading comprehension. The teachers defined assessment as evaluating or measuring students' achievement in the learning process. The teachers also explained that HOTS assessment is a form of assessment to assess students' abilities in higher thinking level, to assess students' critical thinking, and to assess problem-solving level. This was in line with the definition of HOTS assessments by Arter and Jennifer (1987) that the skills used for HOTS assessment are problem-solving (problemsolving ability) and decision-making (decisionmaking).
The teachers also explained that applying and assessing reading using high order thinking skill assessment needed to implement appropriate strategy and method in the classroom. Strategies and methods are an essential part of the teaching and learning process. This can be in accordance with King et al. (2012) claiming that appropriate educational strategies and learning environments promote the growth of higher thinking skills, and promote student tenacity, self-control, an openminded, and flexible attitude. If the teachers implemented proper strategy and method in the learning process and combined them, it would give a good impact on teaching and assessing reading comprehension using HOTS assessment by still considering the students' needs. In addition, the teachers perceived that implemented Discovery Learning could enhance students' higher thinking skills in the learning process. Retnawati et al. (2018) mentioned that resolving complex interdisciplinary problems in group discussions and learning processes is important for training students' HOTS. The strategy and method had to fulfill the students' needs in the learning process, especially when the class consisted of diverse abilities and characters. Therefore, it would achieve the learning objective, and the students were able to master HOTS in reading comprehension.
Regarding teaching reading comprehension using higher thinking skills, the assessment needs to follow what was taught in the learning process. The teachers stated that the assessment must be at the same level as the learning process to examine the students' ability. According to Kaur et al. (2018) this assessment also encourages teachers to think about and develop innovative educational practices that can make lessons more attractive and relevant to students. In addition, the HOTS assessment was appropriate to assess the students' higher level of thinking skills in reading comprehension by providing questions that stimulate the students to think critically. This is in line with Barnett and Francis (2012) that higher-level thinking questions can encourage students to think carefully about the subject. Moreover, the teachers allowed students to evaluate various questions and even applied discussions group. Furthermore, in assessing students' reading comprehension using HOTS assessment, the teachers could implement one kind of assessment based on the need to know students' ability improved. In this case, the teachers chose multiple-choice as their assessment in the learning process. This assessment could be the proper assessment to measure students' higher thinking skills when the teachers applied the rule in designing the questions.
Furthermore, the teachers also got a positive outcome in teaching and assessing reading comprehension using HOTS assessment. HOTS assessment could increase students' critical thinking and improve their abilities. It was also considered beneficial for teachers to engage students' participation in the learning process through discussion activity and enhance their confidence through presentation activity. This is in line with Brookhart (2010) suggesting that the benefits of HOTS assessments are motivation to learn and improved achievement of learning outcomes. In addition, the teachers felt satisfied when the students could overcome their problems and answer the HOTS questions. In summary, both teachers and students positively impacted in applying HOTS in the learning process.

Teachers' Implementation
From the results of document analysis and direct observation were found that the teachers implemented appropriate methods to motivate students to learn reading comprehension using higher thinking skill levels. The teachers employed appropriate activities to find essential information and analyze the text. The activities in the learning process were the implementation of High Order Thinking Skill level through discovery learning. Even though the classroom was held via Microsoft Teams, the teachers still emphasized the students-learning center and gave feedback on what the students had got from the discussion. In addition, the teachers also provided exercises to evaluate students' knowledge about analytical exposition text. This helped students improve their reading comprehension by using higher thinking skill questions.
Compared to the previous studies, this study aligned with some research related to teachers' implementation in teaching using higher thinking skills. Juhansar et al. (2016) explore how teachers used HOTS in the classroom and discovered that HOTS could be applied in questioning and responding to the issues critically, arousing students' class participation actively and accessing the latest information continuously. In addition, Retnawati et al. (2018) stated that HOTS cannot be directly taught to students in the implementation of learning. Students ought to be trained regarding HOTS, as a skill, through learning activities that support its development. Active learning and student-centered learning are activities for training about HOTS. HOTS cannot be communicated directly to students in conducting learning. Students need to be trained in HOTS through learning activities that support their growth. Active learning and student-centric learning are activities for training to become HOTS. Kaur et al. (2017) implied that the teachers' strategies will assist the students to strategize their thinking according to the situation at hand. Instead of just knowing the knowledge, students should manipulate it and turn it into something new and meaningful. Teachers' strategies help students to strategize their thinking according to the situation at hand. Students need to manipulate knowledge, not just know it, to turn it into something new and meaningful.

Teachers' Assessment
Document analysis results revealed that the teachers had good competencies in creating and designing the assessment using High Order Thinking Skill questions. Jensen et al. (2014) stated that higher level skill assessments focused encouraged students to understand the lesson better, apply, analyze, and evaluate but also helped the students to remember better factual knowledge that was instructed by the teacher. They designed the questions properly as revised Bloom's Taxonomy. Both of them developed the questions in analyzing and evaluating level in assessing reading comprehension using High Order Thinking Skill Assessment. However, several questions still used Low Order Thinking Skills. It might be that the teachers focused on students' needs in the learning process. The students have different needs and competencies in gaining knowledge, so the teachers decided to combine LOTS questions and HOTS questions. Furthermore, the teachers could distinguish the questions between level difficulty and level of thinking.
Compared to the previous studies, Wayan (2017) stated that High Order Thinking Skills assessment is an assessment with the following characteristics: measurement of advanced thinking skills (analysis, evaluation, creation), supported discourse problems (typically within the kind of cases), and don't seem to be routine (not familiar). In addition, Arif (2019) implied that teachers need to expand the cognitive process of questions tested against students. A variety of questions can also give you a clearer picture of your students' skills. The various questions also help encourage students to enhance their skills to answer all the questions well. It was also revealed by Afifah et al. (2018) claimed that the role of higher thinking skills in students' test tasks is crucial to increase students' critical thinking. In other words, the questions are not only measuring students' knowledge and focus in memorizing but also their ability to analyze, evaluate and create new ideas so they can increase their logical thinking.

Students' Achievement
The following description is about the results of the students' achievement of HOTS assessment in reading comprehension. The result was gained through the test. This study revealed that after the teachers taught using HOTS in the teaching-learning process, the students had good achievement in reading comprehension using HOTS assessment due to the analytical exposition test. Acosta and Ferri (2010) also used higher thinking skills when reading classes. He found that by using higher thinking skills, students could activate prior knowledge, apply higher levels of thinking, and increase their interest and participation in learning and educational processes. In addition, the mean score in both of the classes was higher than the minimal mastery criterion. Although several students didn't achieve a good score on the reading test using the HOTS assessment, almost all the students got a good score. It proved that the students had different achievements because of their competency to understand the knowledge. Two factors could affect students' competency; external factors and internal factors. The teachers had to observe and find out the best strategy for it. Brown (2004) stated that some aspects such as improving teaching-learning, school accountability, or students' accountability could affect students thinking skills. In addition, the students who answered HOTS questions were as much as those who responded to LOTS questions in this test. It means that the students had good competency to solve and answer the difficult questions demanding HOTS questions. However, the students still must get more exercises to improve their competency in reading comprehension especially using HOTS questions.
The Compatibility of Teachers' Perceptions, Implementations, Assessment, and Students'

Achievement of HOTS Assessment
The teachers were positively aware of HOTS assessments in the teaching learning process.
Their statement about the appropriateness of HOTS assessment in teaching and assessing reading comprehension was proven, and they believed in its good impact. Their beliefs and understanding of implementing HOTS assessment in reading comprehension played an essential role in their classroom practice and professional competency. In the classroom, as an agent of change, teachers need to be forced to recognize, obey, and apply HOTS to teach the skills effectively (Barak & Dori, 2009). More importantly, ensure that you reach the goals of your HOTS related to educational program. Indeed, teacher knowledge and beliefs are the decisive factors (Zohar et al., 2001).
In addition, the teachers were optimistic about conducting HOTS assessments and reading comprehension tests in the classroom. It was revealed when the teachers implemented HOTS assessment in reading comprehension and their perception. The classroom observation helped show the frequency and how the teachers implemented HOTS assessment in the learning process. The findings revealed that teachers' perceptions and implementation of HOTS assessments are aligned with one another. This is following Borg (2006) who believed that teachers schooling, professional coursework, contextual factors, and classroom teaching influence how teachers perceive certain aspects of their field. They used their understanding and lesson plan as their guide in teaching their students. The teachers properly implemented some methods and strategies in teaching and assessing reading comprehension using the HOTS assessment as they stated in the interview.
Furthermore, teachers' assessments showed that the teachers were considered as good in designing and creating the HOTS assessment in assessing students' ability in reading comprehension. The teachers' assessments' findings helped the teachers to design good assessments and develop their professional competency in their practices. The teachers' perceptions and assessments were aligned with one another. They stated that the assessment should align with the topic that had been taught in the learning process. The success of HOTS development is also determined by the coordination between the learning outcomes achieved and the assessments performed at which are closely related to education and learning, as described in the curriculum document (FitzPatrick & Schulz, 2015).
Moreover, the students' reading achievement showed that they had achieved good scores. The students could answer the High Order Thinking Skill question in the test. Preus (2012) explained that students with more advanced thinking activities perform better on high-stakes tests than learning activities that aim for basic skills in a wide range of lessons. The excellent achievement in reading comprehension also indicated that HOTS assessment was an appropriate technique to teach and assess students' reading comprehension. Based on the results, teachers' implementation and assessment were also shown to influence students' achievement in reading comprehension. The findings on students' achievement could help the teachers to evaluate their practices and assessment in teaching reading comprehension using HOTS assessment.

CONCLUSIONS
Based on the interpretation and discussion, two English teachers in SMA N 1 Pati had an excellent perception of the concept, implementation, and benefit of the HOTS assessment. The perception implied from their opinion about the proper concept of HOTS assessment in reading comprehension. The teachers believed that HOTS had an essential role in developing students' critical thinking and solving problems. Moreover, the HOTS assessment was appropriate because the teachers implemented the proper methods and strategies in teaching reading comprehension concerning the needs of the students in the class.
Secondly, the implementation of HOTS assessments in the learning process aligned with the teachers' perceptions and plans to use HOTS for reading comprehension. The teachers implemented HOTS in reading comprehension with the proper method and strategy with different teaching styles. Then, the teachers applied HOTS assessments in the learning process through HOTS questions in the exercises. They designed the learning of HOTS assessment related to students' needs in each class. Therefore, it gave the same opportunity for each student to learn HOTS in reading comprehension and master it.
Thirdly, the teachers' assessment after the learning process on using HOTS assessment in assessing reading comprehension revealed that they designed and created HOTS questions aligned with their perception of the concept of HOTS assessment and included some expert criteria. They also prepared the assessment aligned with what they had taught in the learning process. Despite the teachers preparing the HOTS questions, they still prepared the LOTS question in the assessment. This was because the learning process responded to the needs of the students. Therefore, it found that the teachers used the HOTS test to assess students' reading comprehension correctly.
Fourthly, the students' reading comprehension achievement revealed that they had good achievement in the test. The result of most students achieved the minimal mastery criterion. The students could comprehend and solve the question at the High Order level. They also could understand analyzing and evaluating the High Order Thinking Skill questions. It indicated that the students could master reading comprehension using HOTS assessment in the learning process and the test.
Fifthly, there is a relationship between teachers' perception, implementation, assessment, and students' reading comprehension achievement using HOTS assessment in the learning process. The teachers' perception of using HOTS assessment influenced their implementation in the learning process and their assessment to assess students' reading comprehension. The students' achievement also revealed good result in the two classes. It summarized that the teachers' implementation in teaching and assessing reading comprehension using HOTS assessment was effective, and influenced students' achievement.
Considering the research results, we provide some suggestions for teachers. They should improve their creativity in applying HOTS assessment in the learning process. They can involve various activities to teach reading comprehension using HOTS assessment aligned with the strategy and method. The teachers are also suggested to evaluate their assessment in reading comprehension. They should design more HOTS questions than LOTS questions but still considering the students' needs.