THE EFFORTS TO FOSTER STUDENTS’ SKILL IN MAKING QUESTIONS THROUGH THINKING TOOL (QUESTION MATRIX) DEVELOPMENT

Asking questions and making statements is not easy for students. Many researchers have developed a question pattern, but it has several weaknesses. The question pattern generally has not guided the students to ask low to high level of questions; also, they have no idea what the issue is about and how to naturally propose the question. This research aimed to develop a thinking tool named Question Matrix. This thinking tool intends to train students’ questioning skills in various question levels based on learning purposes. This research was divided into two; small and large scale. The small scale was performed in the even semester, academic year of 2015/2016 while the large scale was in the even semester, academic year of 2016/2017. The method referred to the 4D (Define, Design, Develop, and Disseminate) model. The sample consisted of 104 students. The data were collected through students’ worksheets to collect students’ questions and questionnaires to collect the data of students’ responses to the use of the question matrix. The data collection was conducted in 8 meetings. The quality of students’ questions was scored by using the question quality assessment rubric and categorized according to the revised Bloom taxonomy to determine the percentage of LOT and HOT questions. The research results indicated that Question Matrix had helped the students in making questions. Before the use of Question Matrix, the average small scale research result was 48% (LOT) and 52% (HOT) and increased by 32.50% (LOT) and 67.502% (HOT) after the tool application. In the large scale research, the average cognitive level was 35% (LOT) and 73.5% (HOT). Also, 45.8% of the questions belonged to the ‘Good’ category. In sum, the findings indicated that the Question Matrix is sufficient to be used as guidance in making questions. © 2019 Science Education Study Program FMIPA UNNES Semarang


INTRODUCTION
Asking question is an essential element in the learning process. Questions could help students to learn by linking the first knowledge owned to the new information; accordingly, they could understand what they have learned. Besides, the questions from students play an essential role in meaningful learning and could open their mind to improve the quality of thinking, to un-derstand the concept and to put forward anything they want to know (Almeida, 2012). Question is one of the indicators that someone is thinking. An honest and sincere question asked by students to dig information is proof that students are deeply involved in the learning (Walsh & Sattes, 2011). However, the question asked by the students was still underemphasized (Keeling et al., 2009).
The observation conducted by Ragatz (2010) in Mathematics classes indicates that teachers in Indonesia stated 28 words for every word reported by students, whereas, in other countries, it was only 8-16 words. It means that teachers explain more, while students are seldom to ask anything they do not know to the teacher. However, it does not say that students have understood the material explained by the teacher. Sometimes, some students not understand the material taught by the teacher but have no skill to ask a question. The condition describes that students cannot state what they do and do not know.
Asking a question or giving a statement is not easy for many pre-service teacher students. In the class of Cell Biology in the Biology Education Study Program academic year of 2014/2015 and 2015/2016, there were one to five of the same students who asked questions in one semester, whereas other students did not. Pramudiyanti et al., (2015) report that in the learning Cell Biology course in 100 minutes, students who asked questions were two students.
According to Walsh and Sattes (2011), students were hesitated to ask questions since they did not want to be called as unintelligent. Pramudiyanti et al. (2015) report that making a question was a challenge for students. The condition was due to students who were embarrassed to speak, had low self-confidence to ask a question and needed more time to think about the question. Other difficulties in making a question were that student had less interest to ask a question and the teacher gave only one chance to ask a question in one meeting (Çakmak, 2009). Moreover, students had no understanding of language grammar, did not complete the punctuation, and no relevance between the reading and the question (Pudiyono, 2016).
Many researchers have studied the ability of students to make a question. Pudiyono (2016) stated that the ability to make C2 question by students of English (N=30) with the category of very good was 13.3%. Yuliani et al. (2015) stated that the ability of students to make a question in the material of Reproduction (N=97) was 88.7% that was within the level of C1 and C2. Pramudiyanti et al. (2015) reported that 64.2% of the questions from students were within the level of C1 and C2 based on the revised version of Bloom taxonomy. Coutinho and Almeida (2014) reported that in natural sciences (IPA) learning 45.9% of the questions made by students (N=150) were understanding questions. According to Cardoso & Almeida (2014), 73.3% of questions created by students in Photosynthesis (N=258) learning were modest questions. Keeling et al. (2009) reported that in science education Cell Biology laboratory, 81% of the student wrote LOT questions, and 19% of the student wrote HOT items.
The studies as mentioned earlier were conducted using various methods and approaches in learning. The method used were writing a question (Pudiyono, 2016;Pramudiyantiet al., 2015;Keeling et al., 2009), discourse and discussion (Yuliani et al., 2015), and watching videos, practice, and reading a text (Albergaria- Almeida, 2010;Coutinho&Almeida, 2014). Several ways to improve the ability of students to make a question have been conducted by some researchers, such as through learning model of Reading, Questioning, and Answering (RQA) and method of making a question. RQA model was able to increase questions from students by 13.0% for low-level question (LOT) and 12.7% for high-level question (HOT) (Hariyadi et al., 2017). A method of making a question for six weeks could increase the ability of pre-service teachers to make a question by 26.7% (Yesil & Korkmaz, 2010).
In addition to the methods and models, other researchers have developed tools to train the students to make a question, for example, King (1995) developed a tool in the form of Stem Question, whereas McTighe & Lyman (1998) developed a tool in the form of Thinking Matrix. Llewellyn (2013) presented examples of question patterns based on Bloom's Taxonomy for inquiry learning purposes. King (1995) has designed a method to train students to have critical thinking in Psychological learning by making their questions through a structured question sample and question guide. The guide consisted of a stem question and expected thinking process. Similar to King (1995) asking a question is vital to enhance critical thinking which was explained by Browne & Keeley (2007). Facione (2015), Browne & Keeley (2007) explained that practicing essential thinking can be done by answering questions that have been provided. Meanwhile, Llewellyn (2013) and Kuhn (2010) also provides examples of question patterns to train students to think about inquiry in the learning process.
McTighe & Lyman (1988) have developed a thinking matrix. In the matrix, the vertical part (axis) consists of symbols of thinking types, and the horizontal section comprises of things to be found referring to the object being studied. For example, the use of the matrix in Language and Art subject, teacher and students could use the type of thinking of cause and effect or figure characters in a story. In social science subjects, the idea of the question type and theme or concept can be used.
Based on the description above, the researchers found the weaknesses namely the question pattern is too general, students were not guided to ask questions from low level to high level, and students did not know what the question is about to ask and how to make questions in an easy way, so researchers developed a tool to train students to make questions named the Question Matrix. The Question Matrix completes the thinking tools developed by McTighe & Lyman (1988), King (1995), Llewellyn (2013), Facione (2015), and Bloom's Taxonomy (Anderson &Krathwohl, 2010). Anderson &Krathwohl (2010), Arneson & Offerdahl (2018) divided six categories of cognitive process dimension, namely, remembering, understanding, applying, analyzing, evaluating, and creating. Referring to those categories, the cognitive process of remembering, understanding, and applying are required by students to understand the content/concept of knowledge. Whereas, the cognitive process of analyzing, evaluating, and creating are needed by students to apply the concept of knowledge they owned. Hariyadi et al. (2017) explain that the cognitive dimensions of remembering, understanding, applying, are included in LOT and the cognitive dimensions of analyzing, evaluating, and creating are included in HOT.
The Question Matrix aimed to help students to improve their low order (LOT) and high order (HOT) thinking skills by making their questions. Low order thinking (LOT) is significant for students to understand the concepts of the primary material being studied, whereas high order thinking (HOT) is critical to understand the surrounding situations and problems. (Brookhart, 2010;Collins, 2014). The skill of making a question in low order thinking (LOT) is useful for students to understand the material more simply. A sample of this type of problem is "What is the function of the integral protein in the cell membrane?" When the students can make this question, it is an indication that the students have been thinking about to focus on the topic being studied. Walsh & Sattes (2011) stated that one of the functions of question is to focus attention on things being considered. Besides, the question with topics will facilitate students to use the thinking tool (Arend, 2007). Based on this explanation it is imperative to teach students to make questions using the Question Matrix. The Question Matrix will guide students and the teachers in planning the types of questions needed by referring to desired outcomes learning.
This study aimed to develop a product called the Question Matrix. The Question Matrix developed was expected to be a guide for making questions easily and effectively.

This research belongs to Research and Development. This research developed a thinking tool namely the Question Matrix. The Question
Matrix was tested on 104 students as the research subjects participating in Cell Biology Course in the 2015/2016 academic year and the 2016/2017 academic year at Lampung University. The research instruments included questionnaire sheets for needs analysis, Student Worksheets to write questions, expert validation sheets on the Question Matrix, and Questionnaire sheets to determine students' responses to the use of the Question Matrix.
Product effectiveness testing was performed using a single case study experimental design (Creswell, 2014). The trials were conducted twice, namely on a small-scale (N = 24) and large-scale (N = 80), both experiments were performed in four meetings. Full-scale effectiveness tests were carried out as a means of implementing products that had been packaged in the form of booklets. The test result data namely questions created by students were scored using a rubric assessment on the quality of questions then presented in the way of a percentage.
The research was conducted using the 4D development model (Thiagarajan, 1974;Stes, 2010). The 4D Model consisted of four stages: define, design, develop, and disseminate. The research consisted of four stages following the development model used, namely the 4D model. The first stage is define in which a need analysis was performed. The data were collected by giving questionnaires to lecturers and students. The second stage was design aiming to design the developed product. Thiagarajan et al. (1974) stated that the design stage consisted of activity process of determining the selection of the format and initial design. In this stage, the choice of format was based on the specific characteristics of Cell Biology materials and how to learn the materials. The third stage was develop. In this stage, the expert appraisal was conducted, which is the validation of the product design by the experts. Next was developmental testing, which was a trial of product design on students through small scale and limited scale trials. The validation was performed by three experts of materials, education, and learning. The results were used to improve product development. The fourth stage is disseminate consisting of four steps namely validating testing, packaging, diffusion, and adoption.

RESULTS AND DISCUSSION
In the define stage, the obtained results of the cell biology lecture process were categorized as not good (53.1%), while the curriculum aspect and lecture assessment were categorized good (75.8%) and relatively good (70%). The results of the questionnaire distributed to students (N = 30) showed that students still found difficulties in understanding the material of Cell Biology due to the foreign language textbooks (43.3%) and they had never been taught to make questions to understand the concepts of Cell Biology (80%). Suggestions from lecturers are needed to develop a presentation of learning that can improve material mastery and high-level thinking skills of students. Based on the analysis of the concepts obtained information that the characteristics of Cell Biology material are the structure and function of cell parts, the linkage of the structures and functions of cells, and rarely found in the daily life (Lukitasari, 2013;Martomidjojo, 2011;Pra-mudiyanti et al., 2016). Those terms were, among others the leading strand, lagging strand, SSB (Single Stranded DNA Binding Protein) (Albert et al., 2010;Campbell et al., 2003;Karp, 2010). Based on the results at this stage, it is necessary to develop tools to make questions.
The results obtained at the design stage were two, namely prototype Matrix Questions and rubric assessment of question assessment. Matrix format was used to facilitate making questions from the low level (C1) to high levels (C6) and questions directed to focus on the characteristics of the material being studied.
The Question Matrix developed is presented in Table 1. Blank boxes in the matrix were filled with question codes made by the students. For example, if a student fills an empty box with number 1 in the coordinate [C1,1], it means that the student makes the first question with question level of cognitive 1 (C1) in the material on Facts around us. The result of the assessment rubric consisted of four indicators: cognitive level, grammatical, focus, and relevance. Each indicator was given a score of 1 to 2; thus, the maximum score for each question was 8 points. For the cognitive indicator, score 1 was given if the questions were at C1 to C3, and 2 for the questions at C4 to C6. For the grammatical indicator, score 1 was granted for the items that did not use the standardized language, and score 2 was granted for the items with standardized language. For the focus indicator, score 1 was given if the question was not focused on the concept being studied, score 2 for the questions focusing on the materials. For the relevant indicator, score 1 was awarded for the questions irrelevant to learned concepts; score 2 was awarded for the questions relevant to the concepts. The results obtained in this stage were then validated by the expert at the develop stage.
Validation by experts included appropriateness, feasibility, and effectivity. The results of the expert review on the appropriateness and feasibility were 87.07. This criteria are valid and do not need revisions. The results of this validation indicate that the Question Matrix product can be used for small scale trials. The effectiveness aspects should be tested at the stage of developmental testing. Expert advice, namely an explanation of the definition of each level of questions and examples of questions presented in table form and when using the Matrix Questions have explained the differences in each level of the question.
The results of the rubric validity of the question quality assessment on indicator 1 obtained the Pearson. A correlation coefficient of 0.599, Indicator 2 obtained the Pearson Correlation coefficient 0.806, indicator 3 obtained the Pearson Correlation coefficient 0.811, indicator 4 obtained the Pearson Correlation coefficient 0.717. Referring to Mukaka (2012) the coefficient of 0.5 means that the 1st indicator has a moderate correlation, the coefficient value is 0.7 and 0.8 means that indicators 2, 3, and 4 have a high relationship, so it concludes that the four indicators can be used to assess the quality of questions. The results of the rubric reliability analysis of question quality assessment obtained the value of Cronbach's' Alpha coefficient of 0,714. This value is "acceptable" (Taber, 2017) for assessing quality question.
The effectiveness of the Question Matrix test results is explained as follows. In the first trial (second meeting), the use of Question Matrix obtained information that 13 students (54.12%) knew how to use the matrix, 1 student (4.1%) was still unable to use the matrix, and 10 students (41.7%) incorrectly put the question codes. In the third trial (fourth meeting), information was obtained that 2 students (8.3%) did not make any questions, 19 students (79.2%) knew how to use the matrix, 2 students (8.3%) were still unable to use the matrix, and 1 student (4.2%) incorrectly put the question codes.
The ability students to make the question (Table 2), 2 students had 75 in their grade or had a good quality question before they were using the Question Matrix and after the use of the matrix, the number increased to 11 students (45.8%). Students who had 75 in their grade or had good quality questions usually had a shortage in grammar (punctuation, the use of uppercase and lowercase, and how to write the name of species) and the questions are written were not focused on the material being studied.  The distribution of student questions based on the cognitive level is presented in Table 3. Before the use of Question Matrix, 25 questions were obtained, whereas 80 questions were obtained after the application of Question Matrix. The increase in HOT questions was 15.5% along with a decrease in LOT questions of 15.5%. The use of the question type that referred to Bloom Taxonomy aimed to train the thinking skill of students through the making of simple questions (C1) and advanced to the more profound questions (C4). After the students were able to make a profound question, they could train to create an expanding question (C5 and C6). In Table 3, students' questions before the use of Question Matrix were more in the low order thinking (LOT); however, it improved to the high order thinking (HOT) when Question Matrix was used. In this level, students not only knew about something, but they were able to use the knowledge (Brookheart, 2010).
The questions created before the use of Question Matrix was in the domain of phenomenon and morphology and component. However, it did not occur in the domain of the mechanism. After the use of Question Matrix, the distribution of questions occurred in three domains. The domain of morphology and component still became the most occurred questions. Some examples of question occurred were, among others, the question [C1,2]: "Part of mitochondria that experiences chemiosmosis is called…?" Question [C5,2]: "Why the process of electron transport system occurs in the membrane in mitochondria, does mitochondria excess in the membrane?" Question [C2:3]: "Explain how the respiration process in bacteria?" Based on the above finding, it can be said that Question Matrix could foster the students' skill in making a question. The increase in the ability of students before and after the use of Question Matrix was 15.50%, and it is an indicator that Question Matrix is effective to be used as an aid to make a question.
The disadvantage of Question Matrix developed was in the domain of knowledge 1 (facts around us) and 4 (the example of the phenomenon in the environment), and they had the same meaning; therefore, student questions coded in column 1 and 4 were both correct. As a followup, one of the domains of knowledge, which was facts around us, was used. Question Matrix for Cell Biology, events around us, could use Phenomenon in Organism Body. Thus, the Domain of Knowledge Number 1 was called Phenomenon. The results of the improvement in Question Matrix are presented in Table 4. The final form of the Question Matrix is then packed in a booklet as a supplement to the use of Cell Biology teaching materials. The compiled booklet consists of cover pages, instructions for using the Question Matrix, and Question patterns based on the cognitive Level.
The packaging results were implemented in Cell Biology learning. The implementation results showed that there was an increase in quality before and after using the Question Matrix. Based on Table 5, it appears that the number of students who can make questions in the category of very good and Good has increased. Before using the Matrix Question, there were 47 students making questions in the Very Good category, after using the Question Matrix there were 58 students making questions with a Very Good category or an increase in the ability of 23.40%. The Distribution of Questions based on Cognitive Levels and the Domain of Knowledge showed in Table 6. Before using the Matrix Questions (First Meeting), questions made by students were in the LOT area (81%), and after using the Matrix Questions, the questions were in the HOT area, except at the fourth meeting. The LOT question is an indicator that students have perceptions about Biology courses as memorizing subjects (Momsen et al., 2013). This condition is indicated by the presence of questions about the concept of Cell Biology material. Nonetheless, after using the Question Matrix, the students' questions distributed at various levels of questions and multiple fields of knowledge. There is a distribution of questions that fluctuate between LOT and HOT is reasonable because of the factors characteristic of the material being studied. The distribution of swinging questions is in line with the results of Keeling et al.'s (2009) research namely the production of questions is varying between questions categorized as HOT and LOT. Referring to the number of questions that can be produced by students, the use of the Question Matrix has the potential to improve students' skills in making quality questions. Besides, the questions made by students can be directed to questions that are appropriate to the learning objectives. For students who have difficulties in making questions, they can use question patterns given during the training using the Question Matrix at the first meeting. The difficulties experienced by the students in using the Question Matrix are shown in Table 7, statements number 7, 8, and 9. The difficulties experienced include students who did not understand the level of questions based on Bloom's Taxonomy, students who participated in research samples were students who did not get to know Bloom's Taxonomy. Although the questions have been given based on Bloom's Taxonomy, students still needed some time to get used to using these questions.
One of the students stated about the difficulty in using the Question Matrix as follows "Because I am not used to using the Question Matrix, so it is easier to be spontaneous, but I feel my questions are better after using the Question Matrix." Other students stated that: "The Question Matrix made it difficult for me to make questions because I had to match my mind to the questions I would ask." Based on observations during the learning process, the two students were the trained students who asked questions and had no difficulty in making questions.
We cross-checked the knowledge of largescale sample students using test questions. This test is in the form of C2, and C3 level sample questions and R3 knowledge domain obtained data that 14 out of 80 students answered incorrectly about the question level and 3 out of 80 students answered wrongly about the domain knowledge Question Matrix. Based on this crossexamination, the difficulties in statements 7,8, and 9 can be overcome by getting students to use the question level based on the domain knowledge in the Question Matrix.
The response of students in using Question Matrix is presented in Table 7. Based on Table 7, it can generally be stated that using the Question Matrix can help students think systematically and more efficiently in learning Cell Biology. It means that students were guided in understanding the material being studied and their thinking process was directed. The students were conducted to understand the material, and it is indicated by the response of students in Table 6 that 96.8% of the students were helped in understanding the material being studied. Besides, Question Matrix could help students making a question systematically from the question level C1 to C6. About 94.7% of the students agreed and strongly agreed that they could think systematically due to the matrix. Anderson & Krawthwol (2010) stated that the research results indicated that many students were unable to connect between the facts they learned in class and the events occurred in life. The thoughts can be interpreted that way of thinking is needed to be taught, and one way to do it is through Question Matrix. The matrix will guide students to make a question from the level of remembering to evaluating. The process requires the students first to understand the concept and fact about the topics learned in the class, and then it increases to the use of the concept of knowledge acquired in daily life with a broader scope.
An effort to enhance the skill in making a question can be conducted by giving a way to make a question and teaching the selection of question object (domain of knowledge). Jensen-Vallin (2017) explains that teaching to make a question to students can be done by giving homework in the form of reading quizzes and discussing it in the class. Besides, it can be done by providing understanding to students who have not understood the topic learned, and students are expected to ask a question. Thus, they could realize that asking question is not a sign of weakness or inability, but it is a natural learning process. The Question Matrix can be an alternative tool for teachers and students to practice making LOT or HOT questions. Lemons & Lemons (2013) stated that regularly giving the students a high-level thinking question is very important; it aims to train students to think in high-level so that Biology graduate students are not only able to master the concept of Biology, but also able to provide solutions to the problems faced such as someone who has experienced.

CONCLUSION
After performing a series of 4D procedures, the tool for making questions namely the Question Matrix can be implemented in the learning process. The results of small-and large-scale trials indicate an increase in the quality of the questions made by students. This is an indicator that the Question Matrix is sufficient to guide students to create questions. Students who have used the Question Matrix state that the questions have significantly improved.