A CASE STUDY ON COMPARISON OF HIGH SCHOOL STUDENTS’ SCIENTIFIC LITERACY COMPETENCIES DOMAIN IN PHYSICS WITH DIFFERENT METHODS: PBL-STEM EDUCATION, PBL, AND CONVENTIONAL LEARNING

Literacy skill is needed in the 21 century learning, whereas research about the implementation of Problem Based Learning-Science Technology Engineering and Mathematics (PBL-STEM) to develop students’ ability of scientific literacy is still limited. This research purposed to compare students’ scientific literacy competencies domain improvement through PBL-STEM in the topic of optical instrument. The quasi-experiment non-equivalent group design involved the students of Senior High School 9 Malang Indonesia in three classes called PBL-STEM, PBL, and control class. PBL-STEM class made three products, which were camera obscura, magnifier, and binoculars, which were presented with posters and reports. The PBL class, on the other hand, only made binoculars and presented their work briefly. Scientific Literacy Ability Test was used with 0.88 Cronbach’s alpha reliability. Data analysis with one-way ANOVA, post hoc Tukey test, N-gain, and Cohen’s effect size was conducted. The result showed that three classes had a significantly different ability in scientific literacy. Scientific literacy competencies domain of PBL-STEM was the highest, while PBL class is higher than the control class. The improvement of both PBL-STEM and PBL belonged in medium category, whereas the improvement in the control class was in low category. The effect of the operational implementation of PBL-STEM and PBL pair yielded large result, and both PBL-STEM and control and PBL and control pairs yielded very large category in the improvement of students’ scientific literacy. © 2020 Science Education Study Program FMIPA UNNES Semarang


INTRODUCTION
The human eye, camera lenses, magnifiers, telescopes, and microscopes are known in physics as examples of optical instruments. This topic involves mathematical formulation and technology with an abstract concept to help solve daily problems. Also, it utilizes the scientific concepts both of light ray model, refraction and reflection, and resulted from the implementation of technology and engineering often used to aid us in our daily activities (Young & Freedman, 2016).
Based on Indonesia's revised Curriculum 2013, the topic of Optical Instrument is essential for the three reasons: (1) it is one of the six topics generally taught during the second-half semester of the year, so the students are obligated to learn about it, (2) it is a continuation from the same topic which was learned during Junior High School and Elementary School level of education, (3) it will be learned in great depth in University. However, on National Examination 2018/2019, the understanding level of this topic was only at 30.06% and belonged to the lowest 10% of all topics in Physics. The students mainly have difficulties with light ray due to its abstract nature (Kaltakci-Gurel et al. 2016). Some of them thought the mirror had a real image (Özgür & Küçüközer, 2010). Also, the concept of optical instruments is considered difficult for its uses of mathematical formulation (Rokhmah et al., 2017). This difficultly experienced by the students is also acknowledged by the result of interview of some of Physics teachers, which stated that the topic of Optical Instruments is amongst the three most difficult topics to learn for students. This difficulty may have been resulted from the teachers who gave an abstract explanation in class (Kaya, 2013).
Students' difficulties in understanding the concept of optical instrument can be caused by their low-level of scientific literacy ability. The findings of previous research showed that improvement in understanding physics concepts is affected by scientific literacy (Rustana & Sumantri, 2019), and students who can understand concepts tend to have scientific literacy competencies (Rosa et al., 2018). Scientific literacy on the Programme for International Student Assessment (PISA) in 2015 has three indicators in the competencies domain (OECD, 2016), which are explaining the phenomena scientifically, creating and evaluating the experiments, and interpreting the data collection. Suffice to say the competence domain in scientific literacy deals with how to obtain scientific information, produce and evaluate opinions on the matter, and draw the conclusion based on the scientific evidence (Kuhn, 2010). In a broader range, citizens of a nation should master the scientific literacy to win the competition in the global era (Suryanti et al., 2018).
Students can apply the concept of the optical instrument properly using scientific literacy. However, in the topic of optical instrument, students still have a low ability of scientific literacy (Kurniawati et al., 2019). In one study, students still have scientific literacy indicators in low category (Nur'aini et al., 2018;Rahmayani et al., 2019). Therefore, on a global scale, students still have scientific literacy in a deficient level (Hobson, 2008). This is caused by the manner of learning where students only gain the knowledge of the topic with the emphasis on the mathematical equations without understanding the concepts behind those equations (Rokhmah et al., 2017). The learning that emphasizes the understanding of those concepts is still lacking and causes the misconceptions among the students.
Some researches are dealing with the improvement of scientific literacy for only some of the following topics. Applying an inquiry appro-ach have not yet been able to increase the competence of evaluating and designing scientific investigation in the topic of global warming (Arief & Utari, 2015). Learning with a laboratory experiment in physical geology also proved to be able to train students' scientific literacy (Surpless et al., 2014). Inquiry-based learning-STEM affects students' scientific literacy in the topic of Newtons' law (Yuliati et al., 2018). Inquiry-STEM did not result in any difference in the level of scientific literacy with conventional learning on the topic of Work and Energy (Hudha et al., 2019). Some of the previous research are still lacking on two things: (1) the learning process has not included contextual problems, and (2) there has been no attempt to improve scientific literacy in the topic of optical instruments. This indicates that the efforts in developing students' scientific literacy through learning model that deals with daily problem-solving in the topic optical instrument are still limited.
Scientific literacy of optical instrument is related to abstract principle, mathematical equation, and implementation of technology to help to solve daily problems. Today's learning trend focuses on the development of students' literacy skill, which is one of the many skills needed in 21 st century learning (Jamaludin & Hung, 2017). Meanwhile, there have been proofs that STEM approach can improve the quality of learning process (Guzey et al., 2017) while focusing on solving real problems by working on an unclear daily problem to be a well-solved one by teamwork (Han et al., 2015). STEM approach in all learning process is to improve students' knowledge and skill in a comprehensive manner (Griese et al., 2015) so they can compete in the 21 st century (Becker & Park, 2011). In learning, the application of STEM can aid students to plan, construct, and properly utilize the technology, make improvement in affective, psychomotor, and cognitive skills, and use the knowledge (Kapila & Iskander, 2014). Therefore, STEM approach is suitable with the characteristic of the topic of optical instruments and has the potential to improve students' scientific literacy.
Problem Based Learning (PBL) is a learning model that train learners to think actively to solve daily contextual problems (Potturi et al., 2016). Solving daily problems is the nature of the topic of optical instruments. PBL learning can aid students to increase their science process skills (Gurses et al., 2015), which is a highly positive and meaningful relationship with their scientific literacy (Hasan et al., 2012). PBL supports the growth of scientific literacy indicators, such as the ability to recognize the scientific issue, explain the daily scientific occurrence, and utilize scientific evidence (Ardianto & Rubini, 2016). PBL is one of learning models which is used by researchers to effectively stimulate the growth of students' ability of scientific literacy (Rusilowati et al., 2019). However, PBL needs the basic concept of science to help students in solving a problem (Arend, 2012). The basic concept of science and the process of problem-solving in PBL can be related to STEM. Integrating STEM in learning, we can use designed activities with engineering elements (Tati et al., 2017). In PBL-STEM learning, students are challenged critically, creatively, and innovatively to solve real-world problems through collaborative teamwork (Firman, 2016). However, the implementation of PBL with STEM to develop students' scientific literacy is still limited.
Based on the description above, this study aims to investigate the empirical evidence which shows that PBL-STEM integration has an impact on students' scientific literacy in the optical instruments. The change in students' scientific literacy is detected by analyzing the scores of the PBL-STEM, PBL, and control classes. Then, the effects of the learning process on students' scientific literacy are analyzed.

METHODS
This research used a quasi-experiment with pre-and posttest design (Creswell, 2012). This research was done on three classes in which the students received three different treatments on each class, which are called PBL-STEM, PBL, and control class. All three classes were required to do a pre-test at the beginning, received different learning treatment after the test, and were evaluated with a posttest. The only difference between the three classes was the learning process or treatment. Other things, such as learning hour, curriculum, reference, and teacher, were designed to be equal. The three classes were scheduled on a different day to receive learning treatment, but the hour was equal, which was during the morning. The school implemented the revised Curriculum 2013 and used reference book that was recommended by the government. Three classes were taught by one teacher, which was a pre-service teacher who was on track to finish his thesis on a graduate program at a university in Malang.
The population of this research was all grade XI students of SMA N 9 Malang, Indonesia. This school has a moderate level of students' academic capability. The students were distri-buted into the grade XI randomly. The research samples were 99 students who were chosen with cluster random sampling technique from the existing ten classes. The background of the samples is presented in Table 1.  Table 1 shows that the samples were distributed equally in three research classes. The samples had an almost equal comparison of age and gender.
The learning approaches of the three classes were different. The first class was PBL-STEM. The aspects of STEM were attached to the PBL learning and had a total of 5 syntaxes (Arend, 2012). The first syntax is problem identification. During the second syntax, students are organized to study. In the third syntax, an experiment is conducted to explore the problem. In this syntaxes, aspects of mathematics, technology, and science in STEM were used. In the fourth syntax, students produce and make a presentation on the product. Among the STEM aspects, engineering is the most dominant. Engineering has seven steps, which are identifying problem, looking for all of possible solutions, choosing the best solution, planning and building, testing, revising, and evaluation (Reeve, 2015). The fifth or last syntax of PBL is analyzing and assessing the process of problem-solving, which related to three STEM aspects. The second class only implemented PBL learning. The third class was control class, received conventional method that is consisted of normal activities such as observing a specific phenomenon, listening to a verbal lecture about theory and examples of its occurrences, working on the written problem, and discussing it with the rest of the class. PBL-STEM class has a more complicated process than others because of the presence of the four aspects of STEM which were attached to PBL learning.
Overall, the treatment on PBL-STEM class took a longer time than the PBL class and control class. The learning process was done once a week during two 45 minutes sessions. The learning process of PBL-STEM class lasted for six weeks, with the order of topics as follows:(1) Human Eye and Camera Lenses, (2) Engineering project for Camera, (3) Magnifier, (4) Microscope and Binoculars, (5) Engineering project for magnifier and binoculars, (6) Presentation and evaluation of the result of project engineering of Camera, Magnifier, and Binoculars. The learning process of PBL class lasted for four weeks, with the order of topics as follows: (1) Human Eye and Camera Lenses, (2) Magnifier, (3) Microscope and Binoculars, (4) Activity to make simple binoculars. The learning process of the conventional class lasted for four weeks, with the order of topics as follows: (1) Human Eye and Camera Lenses, (2) Magnifier, (3) Microscope, and (4) Binoculars. It is apparent from the learning process that PBL-STEM class made three products, which are camera obscura, magnifier, and binoculars, whereas the PBL class only made binoculars. Other than that, the PBL-STEM class lasted longer than others, mainly because of the presence of engineering projects. Often students have to study at an extra time outside of the school's schedule to finish the engineering project with the supervision of the teacher.
Every class meeting of PBL-STEM and PBL classes use the same activity in the first and third syntax. The first syntax of problem identification presents daily problems as issues on a worksheet. The topics of the human eye and camera lenses have two issues, which were the blurry picture of the result of the vision of someone with eye disability and a blurry video which contains the result of a photograph of moving object with DSLR camera. The topic of magnifier presented an issue about a video about a man's difficulty to fix wristwatch with the naked eye. The topic of microscope and binoculars presented two issues, which were a picture of a microscopic bacteria which cannot be observed in reasonable means and a picture of a hunter who is having difficulty observing his prey which is far away. These issues will be discussed to find the problem formulation and the alternative solution through the process in the following syntaxes. On the third syntax, an experiment is conducted to explore the problem.
PBL-STEM class made the engineering products earlier than others, so they have more activity in the second, fourth, and fifth syntax compared with PBL class. On the second syntax, students were introduced to the form and examples of the engineering project. Students were directed to prepare the procedure design on the fourth syntax, and also make the planning for the engineering product with the necessary tools for the next meeting. Based on this information, generally, afterwards, students searched and pre-pared the tools outside of the school's schedule with the teacher's control. On this syntax, PBL-STEM class also presented their products with posters and reports, while the PBL class only presented their products briefly. On the fifth syntax, students were shown technology video to add to their knowledge.
The worksheets of PBL-STEM and PBL classes had the same systematic and substance. By order, the systematic of the worksheet is the purpose of the experiment, problem in the form of issue, hypotheses, group investigation, and evaluation of the investigation. However, on the PBL-STEM worksheet, there were columns which were related to every aspect of STEM. For example, the information related to science and technology, the experiments to solve engineering problems, and to verify physics formulas with mathematics principal. This research also had three individual worksheets, which were engineering product worksheet, which contained camera obscura, magnifier, and binoculars. The system of this worksheet was identifying a challenge, exploring ideas, planning and developing, testing and evaluating, and presenting the solution. The worksheet of the conventional class has a simple system, which was the purpose of experiment, tools and ingredients, the steps of the experiments, data analysis, and conclusion.
This experiment used the instrument of scientific literacy ability test. This instrument was presented on multiple issues, and each issue covered one or a few of the test item. This instrument was developed based on scientific literacy competencies domain indicators in PISA question items (OECD, 2016), which was then adapted in the topic of optical instruments. Other than the scientific literacy ability test, lessons plan and worksheet were also made to support the research process.
The instrument was declared as valid based on its construct by experts. Empirical validation was done towards the scientific literacy ability test instrument. The test validation is the test with eight issues which consisted of 14 essays as test items involved 100 grade XII students of SMA N 9 Malang which had learned the topic of optical instrument on the previous semester. The result of the validation was that 13 test items were declared as valid with Cronbach's alpha reliability of 0.88. The distribution of test items into the issues and scientific literacy indicators is presented in Table 2. Data analysis to determine whether there is a difference in the scientific literacy between the three classes was done with one-way ANOVA, to determine which class is better than others was done with post hoc test, and to determine the impact of field operational was done with Cohen's d-effect size (Morgan et al., 2004). N-gain test was done on data to know the improvement of scientific literacy ability (Hake, 2007).  Table 3 contains the category of d-effect size (Morgan et al., 2004) and N-gain (Hake, 2007) of the data of scientific literacy ability improvement

RESULTS AND DISCUSSION
The result of the pre-test score data is presented in Table 1.  Table 4 shows that the mean and standard deviation of each class, respectively, 46.09(7.31), 43.03(6.59), and 42.15(7.66) for PBL-STEM, PBL, and Control classes. It can be seen that three classes have a similar average score in the initial ability of scientific literacy; This shows that students' scientific literacy competencies domain in all classes started with the same condition.
The result of the one-way ANOVA test for pre-test scores is presented in Table 5.  Table 5 shows that there is no difference between classes group. It indicates that the pretest score in PBL-STEM, PBL, and control class was not so different with each other; This means three classes have the same initial ability of scientific literacy. The resulting difference in the scientific literacy competencies domain at the end of the experiment was purely caused by the differing treatments of learning model in all three classes.
The result of the posttest score data is presented in Table 6.  Table 6 shows the mean and standard deviation, respectively, 79.03(15.25), 68.85(12.90), and 54.33(9.18) for PBL-STEM, PBL, and Control classes. It can be seen that students learning with PBL-STEMimproved better scientific litera-cy competencies domain than those learning with PBL or conventional approach. It can also be seen that the students of PBL had better scientific literacy competencies domain than those learning with the conventional method. Based on the result of standard deviation, it is apparent that the scientific literacy score of students in PBL-STEM class was distributed on the more comprehensive range than PBL class, and the students in PBL class were distributed on the more comprehensive range than control class.
One-way ANOVA was used to see the differences in scientific literacy ability in the posttest score. The result of one way ANOVA test can be seen in Table 7.  Table 7 shows that there are differences between group based on Sig. (0.000). Students' scientific literacy competencies domain in three classes is different from each other. Different treatments of the learning model in three classes caused a significant difference in students' scientific literacy.
Post hoc Tukey test was done in posttest data to see if there are differences among the learning methods. The result is shown in Table 8.  Table 8 shows the result of post hoc Tukey for PBL-STEM and PBL, PBL-STEM and Control, and PBL and Control pairs, respectively, are Sig. 0.004-Differences, Sig. 0.000-Differences and Sig. 0.000-Differences. It can be seen that all pairs of classes have a significant difference in the ability of scientific literacy. Based on the average score of posttest, the PBL-STEM class has the highest score, followed by the PBL class and Control class. Based on Table 2, it is known that the three classes have the same initial ability of scientific literacy. Therefore, the result of post hoc Tukey showed that PBL-STEM can improve students' scientific lite-racy competencies domain better than PBL and conventional learning and that PBL can improve students' scientific literacy competencies domain better than conventional learning.
Based on Table 8, PBL-STEM can improve the quality of learning better than PBL or conventional method, and PBL can improve better than the conventional method. PBL-STEM can encourage students to use the science and engineering aspect actively and to gain a deep understanding of mathematics and science to enhance skills and experience in order to use their knowledge directly (Lou et al., 2011). The integration of all aspects of STEM in learning process makes the knowledge to be more meaningful because the students are involved in the process of product design, which enables them to gather, organize, and communicate their findings of the concepts (Torlakson, 2014). The separately taught concepts can then be applied so that they become relevant; This can provide students motivation to learn more (Guthrie et al., 2000). Furthermore, with STEM integration, students are encouraged to pursue their expectation and future dream jobs, and in their attention in science and mathematics (Stohlmann et al., 2012). In short, all gain in interest in PBL-STEMcan lead to gain in students' scientific literacy competencies domain.
Meanwhile, students in the PBL class had more aspect of qualitative problem solving, such as analyzing in choosing the right principles and concepts required in a particular problem (Docktor et al., 2015). PBL model can help students prepare in facing situations and problems in the globalization era (Swan et al., 2013). This research supports the previous finding that the PBL model can improve the ability of scientific literacy of student (Nurtanto et al., 2018) more significantly than conventional class (Suwono et al., 2019). Therefore, PBL learning can improve students' scientific literacy competencies domain.
The improvement of students' scientific literacy competencies domain is shown by analysis of N-gain results which presented in Table 9.  Table 9, it can be seen that the N-gain result in all classes, respectively, 0.611 (medium), 0.453 (medium), and 0.211 (low)for PBL-STEM, PBL, and Control classes. It can be seen that   PBL-STEM is best to improve students' scientific literacy competencies domain, followed then by PBL and conventional learning. This order is the same as the above findings in one-way ANOVA and post hoc test. Furthermore, the N-gain of Conventional class was far below the threshold of average N-gain that can be found in active lear-ning at the score of 0.48 (Jackson et al., 2008). Also, the N-gain score of PBL class was in the proximity of the threshold. The N-gain of PBL-STEM class was able to surpass this threshold.
The improvement of students' indicators of scientific literacy competencies domain is presented in Table 10. Table 10 shows that the N-gain of each indicator in PBL-STEM class is higher than the PBL class, and PBL class is higher than the control class. For each indicator, the PBL-STEM and the PBL classes have a similar N-gain score in the medium category. However, the control class has a different score for each indicator which tends towards the low category. The syntaxes of PBL-STEM or PBL in this research was designed to improve each of scientific literacy indicator in the competencies domain. The first, second, and fourth syntax was designed to improve the indicator explain phenomena. The third and fourth syntax was designed to improve the indicator "create and evaluate an experiment". The fourth and fifth syntax was designed to improve the indicator interpret data collection.
Based on Table 10, PBL-STEM learning can improve the indicators of scientific literacy ability better than PBL learning; This is caused by the presence of STEM aspects, especially technology and engineering, which are lacking or does not exist in PBL learning. The addition of information about technology and engineering activity on second, fourth, and fifth syntax caused the students in PBL-STEM class to be able to improve every indicator of scientific literacy competencies domain. Through engineering process in project exercise, STEM learning can affect students' understanding to make them think they can actively involve themselves in the process of finding the solution, as the designer and maker of products in technology (Berry et al., 2012). Students learned to design the procedure to solve the problem on their own so that product engineering is achieved. The PBL-STEM class made three products, which are camera obscura, magnifier, and binoculars; whereas the PBL class made only 1 product (binoculars). In PBL-STEM class, students tested the products, presented their work result on posters to get feedbacks and improvements if necessary, and made written reports. The engineering application in PBL-STEM class was able to make the students active and acquire in-depth scientific knowledge (Lou et al., 2011); This is different with the students in PBL class which only tested the products and presented their knowledge briefly to get feedback, comments, or criticism. Students in PBL-STEM class were involved in more activities to discover and dig their knowledge to solve the problem. The different treatment in these two classes resulted in a different set of activities and affected students' scientific literacy competencies domain differently. The application of STEM can enrich students' experience through a variety of practical actions (Roberts, 2012). Therefore, STEM learning can evoke students' ability to recognize a concept or knowledge in a problem (Bybee, 2010). The improvement of STEM education can improve peoples' literacy in technology and science (Kelley & Knowles, 2016).
The result of N-gain analysis for each subtopic is shown in Table 11. Table 11 shows that for the subtopic of microscopes, PBL-STEM and PBL classes had lowest N-gain scores; This denotes that this subtopic was considered to be the hardest to comprehend by the students in two classes and following the interview result of few Physics teachers which stated that the subtopic of microscope was considered as the most challenging subtopic by students. The reason is both classes did not build microscope product. The conventional method acquired lowest N-gain score in the subtopic of camera lenses and denoted that this subtopic was considered to be the hardest to comprehend by the students in Control class. The N-gain result of all subtopics from the highest order to the lowest order is PBL-STEM, PBL, dan Control classes. The cause is PBL-STEM class made three products with the posters and reports, PBL class made only one product, and control class did not make any product. All classes have the highest N-gain on the subtopic of magnifier because this subtopic is more familiar to the students than other subtopics.
Analysis of the effect size in PBL-STEM, PBL, and Control class is presented in Table 12.  Table 12 shows that the results were respectively, 0.721 (Large), 1.962 (Very large), and 1.297 (Very large)for PBL-STEM and PBL, PBL-STEM and Control, and PBL-Control pairs. It shows that the implementation of PBL-STEM has a higher effect of improving students' scientific literacy competencies domain compared with PBL. However, the effect size of PBL-STEM and Control class pair and PBL and Control class pair belong in the very large category. It shows that the implementation of PBL-STEM or PBL learnings has a much higher impact on improving students' ability of scientific literacy compared with conventional learning (Morgan et al., 2004).

CONCLUSION
From the result and discussion, few conclusions can be drawn. The three classes had significantly different students' scientific literacy competencies domain. Based on the average score of posttest, the PBL-STEM class has the highest score, followed by PBL class and Conventional class.PBL-STEM learning can improve students' ability of scientific literacy better than PBL learning and conventional learning classes, and that PBL can improve students' ability of scientific literacy better than the conventional method. Also, the improvement of both Experiment A and B classes belonged in medium category, whereas the improvement in Conventional class was in a low category. The effect of the operational implementation of PBL-STEM and PBL pair yielded "large" result, and both PBL-STEM and Control and PBL-Control pairs yielded "very large" category in the improvement of students' ability of scientific literacy