An Analysis of Higher Order Thinking Skills Realization in Reading Comprehension Questions

The significance of Higher Order Thinking Skills (HOTS) has led many educational stakeholders to include the skill in educational curriculum and more specifically in the subjects that students learn. In the context of learning English, HOTS should be instilled in four language skills, especially reading skill. This skill is vital since students are demanded to be critical of what they read. Therefore, this research aimed at investigating how the reading comprehension questions in English textbooks for three different grades (X, XI, and XII) published by the Ministry of Education and Culture facilitate HOTS. By employing cognitive level of Bloom’s revised taxonomy (2001) and critical thinking task type of Numrich’s sequence (Beaumont, 2010), the questions are analyzed qualitatively. The results reveal that HOTS are realized adequately in the textbook for grade XI. In the textbooks for grade X and XII, HOTS are found in small quantity. Besides, based on Numrich’s sequence, the majority of the questions focus on the text which means that critical thinking-based questions are found in small number. Thus, it is expected that this result assists teachers, textbook writers, and future researchers to engage more on the implementation of HOTS.

thinking. It is explained that HOTS are defined into three categories namely transfer, critical thinking, and problem solving (Brookhart, 2010, p.3). Based on this categorization, it is defined that HOTS in terms of transfer demand students to apply knowledge and skills they learn in a new context that they do not repeatedly experience. In terms of critical thinking, HOTS encourage the skill of making judgement and making decision. While on the problem solving, students are expected to formulate solutions to certain problems. Therefore, in accordance with the definitions, HOTS can be assumed as skills of engaging with unfamiliar context, the activity of making judgement and decision, and also solving problems. In connection with the categorization, it is inferred that through the emphasis of HOTS, there is an effort of promoting critical thinking. In connection with the insertion of HOTS in the educational field, it directly affects the design of teachinglearning activities and textbooks in facilitating HOTS. It is expected that those components provide wide opportunities for students to exercise HOTS. Moreover, textbook takes an integral part in promoting HOTS since it becomes a universal element of ELT teaching (Hutchinson & Torres,

INTRODUCTION
The great challenges and rapid development of science and technology demand people to possess the skills needed in this era in order to win those challenges and demands. The skills are formulated under the name of 21 st century skills which include critical thinking and problem solving, creativity and innovation, collaboration, question formulation, global awareness, communication skills, and technology skills (Harvard Advanced Leadership Initiative, 2014). Those skills are perceived to be the provisions to face the global challenges in these days.
Noting the significance of the skills, there is an initiation of the inclusion of the 21 st century skills in many educational curricula. This attempt is also projected in the Indonesia national education curriculum which emphasizes Higher Order Thinking Skills (HOTS) as one of the competencies that are envisioned (Pusat Penelitian Kebijakan Pendidikan dan Kebudayaan, Badan Penelitian dan Pengembangan, Kementerian Pendidikan dan Kebudayaan, 2018). The inclusion of HOTS is believed as the effort of fostering critical thinking since it is in line with the statement by Brookhart (2010) regarding the relation between HOTS and critical can practice, see their progress, and even develop HOTS. However, based on my previous studies, it is found that HOTS-based questions are realized in small quantity. Another result obtained by Ulum (2016), Barashid (2020), Tangsakul (2017), andPermatasari (2012) revealed that most of the reading comprehension questions are tapped by the lower categories and less likely promote higher categories. Ulum (2016) who conducted a descriptive content analysis of a reading comprehension questions in a course book Q: skills for success 4 reading and writing according to Bloom's taxonomy revealed the same findings that the reading comprehension questions were lack of HOTS. These results urge the action to assure the implementation of HOTS in reading comprehension questions.
The results of some previous researches bring us evidences that textbooks play an important part in supporting the development of HOTS, and reading comprehension questions should ideally implement HOTS. However, based on previous studies HOTS are realized in small number. Connecting these facts, it is essential to make sure that the elements of teaching-learning activity, such as, textbook and reading comprehension questions should implement HOTS. Therefore, this present research is intendedly conducted to analyze how HOTS are realized in the reading comprehension questions found in English textbooks using cognitive level of Bloom's revised taxonomy (2001), and task types of Numrich's sequence (Beaumont, 2010). The research is mainly formulated to answer five main questions: (1) how are HOTS realized in reading comprehension questions found in the textbook for grade ten?; (2) How are HOTS realized in the reading comprehension questions found in the textbook for the eleventh grade?; (3) How are HOTS realized in the reading comprehension questions found in the textbook for the twelfth grade?; (4) How are the reading comprehension questions found in the textbooks for the tenth, eleventh, and twelfth grade realized critical thinking task of Numrich's sequence?; (5) How are the differences among those three textbooks in realizing HOTS and critical thinking of Numrich's sequence in reading comprehension questions?

METHODS
This research is a content analysis analyzing the content of textbooks, specifically the reading comprehension questions found in the analyzed textbooks. Since this present research aimed at describing the analysis result regarding that exist not only for students, but also for teachers. For students, textbook provides several components such as materials, activities for students' practice, and syllabus (when reflected the objectives that have been determined) (Cunningsworth, 1995, p.7). In addition, textbook also provides components that ease teachers in deciding what the most appropriate things for their classes are (Bojanic & Topalov, 2016). This is strengthened by Herlinda (2014) stating that textbooks play a crucial role in teaching-learning activity. It is revealed that both teachers agreed that textbook provided materials and activities aspects in language learning, guided teachers in giving homework and created standardized instruction, and also gave idea in making lesson plan. It was also found that in teaching-learning activity, both teachers use more than one textbook as reference. This shows that textbooks play a vital role in classroom activity.
Realizing those significant roles; therefore, textbooks should ideally come with materials and exercises fostering HOTS in any subjects, including English. HOTS should be realized in all of the language skills, specifically reading skill since it is extremely required nowadays. Moreover, in current days, students are not only demanded to be able to read, but are also demanded to be able to make meaning of the information they gain (Wahyuni, 2019) and further to be critical of what they read (Taylor, 2018).
Referring to the importance of making meaning from reading text; thus, it is no wonder that great effort of assessing and exploring students' performance on the reading skill has been made. For instance, international test, namely PISA, has been conducted in order to assess proficiency on reading, mathematics, science and an innovative domain, and students' well-being (OECD, 2018). Referring to the result of the test held on 2018, it is revealed that Indonesian students scored lower than the average score of OECD in reading. Translating the score, 30% of Indonesian students attain at least level 2 of proficiency which primarily deal with identifying main idea in a moderate length, find information based on explicit, though sometimes complex criteria, and reflect on the purpose and form of texts when explicitly directed to do so (OECD, 2018). Some a negligible percentage reach top performers. These impliedly means that further attention on the reading proficiency should be put.
Taking the issues into consideration, there should be created exercises and specifically sample of reading comprehension questions promoting HOTS. By providing these aspects, students the realization of HOTS in reading comprehension questions using cognitive level of Bloom's revised taxonomy (2001) and critical thinking task types of Numrich' sequence (Beaumont, 2010), it is categorized into descriptive qualitative research.
The reading comprehension questions under the study are those of questions following reading passages found in the textbooks entitled Bahasa Inggris SMA/MA/SMK/MAK kelas X, XI, XII Edisi Revisi 2017 published by the Ministry of Education and Culture. There are 170 reading comprehension questions in the textbook for grade X, 42 reading comprehension questions in the textbook for grade XI, and 117 reading comprehension questions in the textbook for grade XII which were examined. Since the data under analysis are in the sort of written text, thus, it is categorized as qualitative data.
In collecting the data, documentation was chosen with three main steps: (1) reading the textbooks under analysis; (2) collecting the reading comprehension questions; and (3) documenting the questions.
Having been collected, then, the data were analyzed. In analyzing the data, a framework of data analysis by Miles and Huberman (1994, p.10) was employed. Accordingly, there are three major steps of data analysis: (1) data reduction; (2) data display; (3) conclusion and verification. Data reduction is the stage of organizing the raw data into the data which are ready for further analysis and conclusion drawing. Data display is believed as the stage of displaying the compressed data. Lastly, conclusion and verification is the stream of concluding the displayed data and then verifying the conclusion. Referring to the framework, the data on this present research were analyzed with the following procedures: (1) sorting relevant reading comprehension questions and listing the questions based on the textbook they belonged to; (2) identifying the data and displaying the result of identification in the form of table and narrations; (3) drawing conclusion and validating the conclusion.
In order to validate the conclusion, triangulation was conducted. It is employed in order to show that other independent measures support the findings or at least don't contradict (Miles & Huberman, 1994:266). In doing triangulation, triangulation by data source and theory were employed. Therefore, two English teachers were invited to share their perspectives on the implementation of HOTS in reading comprehension questions in English textbooks trough questionnaire, and related previous researches were selected to triangulate the analysis result. The procedure of analysing data is visualized in Figure 1.

RESULTS AND DISCUSSION
As previously mentioned that this research specifically investigates the realization of HOTS in reading comprehension questions in three English textbooks for different level of grades using cognitive level of Bloom's revised taxonomy and critical thinking task types of Numrich's sequence (Beaumont, 2010), it is revealed that HOTS are realized in three different ways. In general, HOTS are realized in the three textbooks, however, the realization is in different number. For more detail result, Table 1 visualizes the frequencies and percentages from the three textbooks.
Citing the frequencies and the percentages of the realization of cognitive level shown in Table 1, there are three main points that can be derived: first, in the textbook for grade X, the Although applying category is not found, remembering and understanding are found in high percentages. Remembering becomes the major category found with more than three quarter of the total questions. In dealing with this category, students mostly cope up with explicit information written in the text, e.g. Does she like animals? What impression do you get when you read the word majestically? In accomplishing the questions, students only need to read the text and scan the related information. The answers are explicitly written in the text. This type of question is found in more than three quarter of the total questions, which means that students mostly deal with the question focusing on the lower category. This phenomenon makes the realization of the higher categories in lower percentages. Linking to the frequencies of HOTS-based questions, there are only 13.52% of HOTS based questions. These HOTS-based questions are distributed into analyzing 9 questions, evaluating 11 questions, and creating 3 questions. Through those type of questions, students are encouraged to deal with the skill to analyze information on the text by differentiating the elements found there, such as, Read the text 1 again. Find out the similarities between text 1 and text 2. This question demands students to break the text and analyze the elements to find out the similarities and differences. Another skill to give evaluative judgement is also manifested in the question like, Do you think that the information in the text is clear? This question requires students to evaluate the information in the text and then give their judgement on that information based on the imposed criteria. An important skill to create is also emphasized through 3-unit questions. In the skill of creating, students are demanded to be able to hypothesize, make a plan, or create a product, for instance, Why do you think the date of the battle of Surabaya is used as a momentum to commemorate our hero's contributions? Through this question, students need to deal with the process of hypothesizing premises.
Second, the reading comprehension questions found in the textbook for grade XI are dominated by HOTS. This is proven by the availability of HOTS in more than 50% of the questions. Based on the analysis, there are 61.9% of the reading comprehension questions which are categorized into HOTS. There are 26.19% of analyzing category, 28.57% of evaluating category, and 7.14% of creating category. From the three categories, evaluating is found higher than analy-zing and creating. In evaluating, students mostly deal with making judgement and checking inconsistency. This skill becomes one of the main concerns in critical thinking in which students see an issue from various perspectives and positions (Parviz & Imam, 2018). This skill is important for the development of critical thinking for students. Hence, a great number of questions in this category could stimulate and help students train HOTS ability.
The great number of HOTS-based questions proves that students are given wider opportunities to practice higher thinking skills. This can be seen through the manifestation of analyze, evaluate, and create in a more than half of the total questions. For instance, these skills are found in the questions, for examples, Do you think the poet is content with his choice? Give reasons to support your answer!; Do you think smoking is dangerous?; Do you think it should be banned?; If you had to change the lyrics of stand by me, which lyrics would you change?; Who is your hero?; What might the two roads represent or symbolize? Make a list of possibilities and discuss with your partner! These questions promote the process of investigating writer's point of view, making judgement, and hypothesizing. These processes become one of the concerns mainly emphasized in higher order thinking skills.
Third, reading comprehension questions found in the textbook for grade XII are dominated by lower category, which is remembering category. This creates a situation in which the higher categories are only attained in small quantity. From overall 117 questions, HOTS-based questions only appear in 19 out of 117 questions or in 16.2% of the questions. These questions are distributed into analyzing 8.54%, evaluating 5.12%, and creating 2.54% of the questions. In comparison to LOTS, HOTS are found in lower percentages.
In HOTS-based questions, the skill is translated into the question such as What is being debated? What is the fact? Why do you think so?; What differentiate one type of bullying from another?; do you think is the most damaging kind of bullying?; If you were one of the parents, what would you do to respond to some problems with the online system? In the first and second questions, students are required to demonstrate the skill of analyzing by distinguishing facts from opinions, and also differentiating one element to another. While the third question, students are given a challenge to give judgement over kind of bullying. What students are expected to do is deciding which kind of bullying that fits the label of the most damaging, considering all the features mentioned in each type of bullying. On the other hand, the last question asks students to formulate solutions to solve online system problem. In this task, students must produce a solution to overcome the problems. This task belongs to creating category in which students mainly focus on connecting elements to create a functional whole (Anderson & Krathwohl, 2001, p. 84).
The cognitive level of Bloom's revised taxonomy (2001) is created for the purpose of assisting teachers in designing tasks based on the cognitive level that is aimed to be reached (Anderson & Krathwohl,p.6). The categories on the cognitive level work hand in hand. The lower categories need to be emphasized to support the mastery of the higher categories. The higher categories are those the skills of analyzing, evaluating, and creating. In relation to that, only reading comprehension questions in the textbook for grade XI provide adequate number of questions that are qualified to HOTS. This is claimed as adequate since the realization of HOTS is found in more than 60% of the questions. The frequent number of HOTS gives students more platform to sharpen their ability on solving problems requiring higher degree of thinking.
However, the reading comprehension questions found in the textbooks for grade X and XII, majorly focus on the lower categories, especially remembering. This category is good to recall the previously learnt information, but too many questions of this category is not enough to promote HOTS. In these two textbooks, HOTSbased questions are only found in less than 20% of the questions which is not sufficient to stimulate and further progress the ability of HOTS. This brings the evidences that HOTS ability should be developed. This development can be realized by familiarizing the skills for students. One thing that can provide the platform to familiarize and train the ability of HOTS is textbook (Pratama & Retnawati, 2018). Thus, in fostering HOTS for pursuing critical thinking as how it is mentioned in the 21 st century skills, HOTS-based questions should be ideally realized more often (Wu & Pei, 2018).
To sum up, two major findings emerge from this present research. The reading comprehension questions found in the textbooks for grade X and XII mostly deal with the lower categories than the higher categories. HOTS-based questions are presented in a small number of questions. These are in line with the result obtained by Ulum (2016), Brasahid (2019), and Tangsakul (2017. In those researches, reading comprehension questions are still lack of HOTS-based questions. In comparison, there is an initiation of imposing HOTS in reading comprehension questions in adequate number. This is realized in the reading comprehension questions found in the textbook for grade XI. This is supportive with the research by Setiyawati who unveiled that HOTS are found dominant in the textbook she investigated. These two findings are confirmed by two English teachers, who are the users of the textbooks. They stated that HOTS are moderately realized in the three textbooks. This means that HOTS, are realized, but not in sufficient number. Aside of HOTS analysis, the analysis based on task types of Numrich's sequence is also conducted. The result of the analysis is displayed in Table 2. Table 2 shows the result of analysis based on task types of Numrich's sequence. There are seven tasks which are categorized into three perspectives. Those tasks are the sequence in building critical thinking starting from investigating personal views, engaging with the main text, and focusing beyond the text (Beaumont, 2010).
From Table 2, it is presented that the task types are not equally realized. From the three textbooks, observing and inquiring further are not found. In contrast, understanding and organizing, and interpreting are dominated. Understanding and organizing task is found in more than 50% of the questions in the textbooks for grade X and XII. Besides, interpreting task is realized in nearly 50% of the questions in the textbook for grade XI. The domination of understanding and organizing, and interpreting shows that most of the questions lead students to focus on the information on the text. In understanding and organizing task, the answer can be directly picked from the explicit excerpt on the text. This task is part of critical thinking, however, in dealing with the task, students do not need to think critically since the answer is already written on the text. This is slightly different from interpreting task. This task demands students to focus on the information in the text, but students are required to understand the information deeper. They do not rely on the literal statements, but they have to investigate the meaning behind the literal information. However, in this task, students still focus on the text and not yet move to think beyond the text.
Seeing the percentages, it can be stated that the tasks focusing beyond the text are distributed in lower percentages. In the textbooks for grade X and XII, questions focusing beyond the text are only found in less than 10% of the questions. Moreover, inquiring further is not realized, and making decision is only found in 2 question items. This also happens in the textbook for grade XI. However, in this textbook, the percentages of the activity focusing beyond the text is higher with 26.18% of the questions.
Those representations share similarity in terms of focusing on the text. The difference is on the task the questions mostly focus on. In the textbooks for grade X and XII, understanding and organizing task is dominating. In the textbook for grade XI, the majority of the questions belong to interpreting.
Deducing the information related to the task types of Numrich's sequence in the three textbooks, it is discovered that the reading comprehension questions primarily focus on the text. This perspective is surely part of critical thinking; however, dealing with this perspective too much is not enough to stimulate critical thinking. Moreover, focusing on evaluating, analyzing, making decision, and problem solving, which are important in critical thinking, are only found in small quantity.
According to Paul and Elder (2008) citing in Parviz and Imam (2013) critical thinking is a matter of analyzing and evaluating in the purpose of improving it. Critical thinking is a kind of thinking involved in solving problems, designing inferences, calculating likelihoods, and making decision (Parviz & Imam, 2013). In comparison to the definitions, from three analyzed textbooks, the task of analyzing, evaluating, finding more sources of information, and making decision are found in small percentages. Inquiring further, which focuses on finding out more information is not found in the three textbooks. Analyzing and evaluating is found in less than 6% of the questions in the textbook for grade X, 21% of the questions in the textbook for grade XI, and less than 7% of the questions in the textbook for grade XII. Furthermore, making decision task is only found in 2 questions in the textbooks for grade X and XI, and 3 questions in the textbook for grade XII. Inferring the percentages, the critical thinkingbased questions are found in minimum percentages. In comparison to that, critical thinking seems to be imposed in only small quantities and needs more number in order to pursue the skill.
In connection to critical thinking task types of Numrich's sequence, it is explained that critical thinking is built by experiencing various activities that give increasing challenges. It starts from investigating students' own world, focusing on the text, and then focusing beyond the text (Beaumont, 2010). It is also explained that critical thinking activates emotional or get responses to finally create a reasonable judgement (Beaumont, 2010). However, it is revealed that the reading comprehension questions only focus on the text. The activity of investigating students' personal emotion and then connect it to think beyond the text is somewhat neglected. Thus, it can be said that the three textbooks are less likely boost critical thinking since the tasks that students perform do not call students to think critically to engage with the world beyond the literal information that they gain in the text and the problem beyond what they solve in the text.
Those results are agreed by the findings obtained by Permatasari (2010) which released the information regarding the tasks students mainly focus on. In that research, the majority of the questions push students to focus on the text. This means that the attention on developing critical thinking-based questions should be increased.

CONCLUSIONS
In accordance with the presented findings, it is drawn that HOTS are only adequately realized in the textbook for grade XI. In the textbooks for grade X and XII, HOTS are found there, but in small quantity. This is quite surprising since HOTS-based questions are demanded in current days by the curriculum. Furthermore, it is also explained that critical thinking-based questions according to task types of Numrich's sequence appear in low percentages, for the questions commonly focus on the text. Therefore, taking the findings into consideration, it is advisable that education stakeholders, teachers, textbook writers, and house of publishers should give more attention on the effort of fostering and developing HOTS. It is expected that more sources on HOTS-based questions should be added. Further action in engaging with the issue on the development of HOTS are expected to be initiated.