IMPACT OF INTEGRATING SCIENCE AND ENGINEERING TEACHING APPROACH ON STUDENTS’ ACHIEVEMENT: A META ANALYSIS

STEM teaching approach has attracted the attention of science educators as a panacea in addressing the deteriorating interest in STEM learning. However, the impact of STEM teaching approach on students’ achievement is scarcely discussed. Thus, this study aimed to explore the impact of integrating Science and Engineering teaching approach on the achievement of students. This meta-analysis was conducted systematically on articles published between January 2000 to January 2020 which were retrieved from five databases (EBSCOhost, Emerald, Scopus, Web of Science and ERIC).  Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) were applied to ensure a rigorous selection of the desired research articles. The data were then assessed using two online statistical effect size calculators. A total of 14 effect sizes were calculated. The overall mean effect size obtained was of high effect (es=2.61).  The Engineering Design Process (EDP)(es=3.26, high effect) was the most effective teaching approach in enhancing students’ understanding, as opposed to design-based learning.   An implication of the study is that EDP should be the basis for other teaching approaches in SE. Solving and creating tasks encourage application of concepts that then lead to enhancement of achievement.


INTRODUCTION
A curriculum consisting of science, technology, engineering, and mathematics, or generally mentioned as STEM, was first conceptualized by educators in the United States almost three decades ago. This came about because of the scarcity of college graduates in relevant technical occupations following the rapid growth of scientific advances in the late 1990s (Vasquez et al., 2013;Chesky & Wolfmeyer, 2015;Martin Paez et al., 2019). Later, academics globally realized the need to improve STEM education to equip students with the necessary STEM skills and knowledge for future job requirements. White (2014) explained that STEM education was an initiative created by the National Science Foundation (NSF) and was originally and was originally called Science, Mathematics, Engineering and Technology (SMET). However, current research suggests that STEM education can be branched off into another perspective, namely integrated STEM education (Martin Paez et al., 2019). While the classic STEM education focused on the silo approach of four STEM components mentioned previously (Kelly & Knowles, 2016;Permanasari et al., 2021), integrated STEM education emphasizes the different levels of integration and interconnections between the subject areas as well as the rich contextualization of content through real-world applications (National Research Council, 2014).
In general, integrated STEM education need not include all four STEM disciplines. Previous literature suggested that it can be described as: (i) an approach that integrates more than two STEM subjects; (ii) an integration of engineering design; and (iii) an attempt to incorporate all four STEM components into a real-world problem where teaching is performed in an authentic context (Moore et al., 2014;Kelly & Knowles, 2016). As indicated in previous studies, integrated STEM education carries various definitions without any consensus among scholars. There were three important points about its integration. Firstly, integration does not mean abandoning standards but describing the skills and concepts within the disciplinary areas. Secondly, the power of STEM lies in students realizing that solving problems can be informed by various disciplines. Thirdly, the integrated STEM curriculum enables students to learn to apply STEM concepts and skills in their daily lives.
An 'Integration Continuum' curriculum presented in Vasquez et al.'s work (2013) defined STEM integration as a continuum of accumulating stages of integration in which progression across the boundary involves extensive interconnectedness and interdependence between the disciplines. To summarize, the progression starts with a disciplinary approach whereby students learn concepts from separate disciplines before learning them within a mutual theme at a multidisciplinary stage. Afterwards, in order to deepen knowledge and skills, interdisciplinary learning is organized across disciplines. Finally, during transdisciplinary approaches, students get to apply their skills and knowledge through projects or problem-based learning (PBL) in real-world applications.
In this study, the integration of Science and Engineering (SE) is located in the third level (interdisciplinary integration). Once the disciplines are interconnected and interdependent, it allows the mastery of concepts and skills to be accomplished without neglecting the fundamental knowledge of STEM disciplines. In addition, it also helps one, particularly students, to understand that different disciplines can be combined for problem-solving and application in everyday life. In this regard, researchers discovered the PBL approach to be a dominant teaching approach for integrated STEM implementation (Mustafa et al., 2016;Kang, 2019). However, PBL in science education is different from PBL of integrated STEM education (Puente et al., 2013). PBL in science education usually involves scientific practices. Meanwhile, for integrated STEM education, PBL activities include both scientific and engineering practices, namely the Engineering Design (ED) process. ED is a teaching approach that constitutes a learning process that draws upon the sciences, mathematics, and engineering knowledge to meet the desired engineering outcome (Baker & Galanti, 2017). In addition, Shahali et al. (2016) noted that ED is not only limited to solving engineering problems but is also beneficial for creating new technologies and products.
Furthermore, for the last decade, studies have also shown that the ED process in learning science or mathematics can generate a meaningful outcome through the implementation of scientific and mathematical principles to solve complex and real-world problems (Bryan et al., 2016). Additionally, ED enables students to enhance their cognitive abilities, namely problem solving, creative thinking, formulating solutions, and decision-making skills. As a result, students would be able to learn mathematical or science concepts through a more authentic process by relating them to real-life situations (Moore et al., 2013;Banks & Barlex, 2020).
The different concepts of ED can be divided into three main perspectives, namely pedagogy, disciplinary practice, and core idea. Purzer (2017) even mixed these three perspectives. These variations are influenced by three different research movements in the United States over the past 30 years. In the late 1990s (first research movement), learning design was perceived as a pedagogical approach focusing on the disciplinary core ideas in physics and biology subjects. These pedagogical approaches are also called as follow: (i) learning by design; (ii) design-based learning; and (iii) design-based science. Meanwhile, the second research movement was led by educators in the engineering field using two influential publications, namely: (i) a consensus study report titled "Engineering in K-12 Education: Understanding the Status and Improving the Prospects"; (ii) a report on the status of engineering education in the US (National Research Council, 2009). Consequently, engineering was then framed as a core content while the design was promoted as a disciplinary practice. Now, integrated STEM emphasizes this effort as a pedagogical approach and the design as a practice. The current research movement focuses on ED as a mixture of the three perspectives, consisting of instruction, disciplinary practice, and fundamental ideas.
ED as a pedagogy can be further divided into three approaches, namely Learning by Design (LBD), Design-based Learning (DBL) and Design-based Science (DBS). The three approaches vary in terms of the learning process, knowledge acquisition and learning outcomes. LBD focuses more on problem-based and case-based learning, making it easier for students to experi-ence meaningful learning of science and design practice. In contrast, DBL accentuates comprehension of abstract concepts by integrating ED and science inquiry (Apedoe et al., 2008). As for DBS, it is more of an inquiry-based approach where the focus is on engaging students' metacognition to enable them to present ideas in multiple ways and improve their problem-solving skills, resulting in learning outcomes in the form of engineering products (Fortus et al., 2005).
ED as a pedagogy does not represent the epistemological underpinnings of each field. Therefore, another framework is used to overcome these dilemmas, namely ED as a Disciplinary Practice. In this practice, design-based learning is implemented. For instance, the novel engineering approach is implemented at the elementary level (Purzer, 2017). This innovative approach integrates characters from books and novels for students' understanding to engage them with the Engineering Design Processes (EDP). STEM unit's integration is another approach which engages students in authentic disciplinary practices (Mathis et al., 2016). According to Goldstein et al. (2017), in implementing STEM unit's integration, the collaboration between educators and stakeholders is considered the best approach at the secondary level because it enhances students' interest in establishing a career in STEM.
Meanwhile, ED as a core disciplinary idea is defined as a framework which outlines two disciplinary core ideas that associate engineering with technology, such as the implementation of the Engineering, Technology and Application of Science (ETS) framework for K-12 Science Education. ETS 1 emphasizes on "How do engineers solve a problem?", while ETS 2 is about "How are engineering, technology, science and society interconnected?". This approach is introduced in primary education and is implemented throughout high school (NGSS, 2013).
Previous literature has indicated that two factors affect the implementation of integrated STEM teaching approach, namely teachers' beliefs and students' beliefs (Dong et al., 2020;Vallera & Bodzin 2020;Arshad et al., 2021). Teachers' belief in their students' learning ability is important for effective SE integration in the teaching style and it also influences students' achievement in school (Margot & Kettler, 2019;Christian et al., 2021). According to Wang et al. (2011), besides emphasizing on educational problems, curriculum, and techniques to bring the topic alive for students, teaching also requires pedagogical content knowledge. Each teacher possesses a specific skill set in forming the material and pedagogical methods which he or she plans to implement in a classroom.
Moreover, the expertise in integrating STEM into instructions is related to the teachers' ability, teaching background, content knowledge, time, curricular aims, assessment, and classroom management skills (Gresnigt et al., 2014;Christian et al., 2021). In addition, teachers' beliefs and views about teaching and learning, which might entail resistance or lack of motivation to change their practices, are some of the factors that may impede the implementation of integrated STEM education (Ashgar et al., 2012). Thus, in this study, we conceptualized teachers' beliefs as their capabilities in integrating SE to produce the desired effect on student learning. However, the inclusion of engineering knowledge in science classes has led to problems such as a lack of confidence amongst teachers as they are not equipped with the requisite engineering knowledge to teach students (Kim et al., 2019). Nevertheless, despite teachers' lack of teaching knowledge, teachers frequently use their own pedagogical content knowledge and practice to integrate SE.
The second factor affecting the implementation of integrated STEM teaching approach is students' beliefs. Currently, not much educational research has been carried out on students' belief in the learning process (Halim et al., 2014). While every teacher or student has their own perspective in conceptualizing integrated STEM education, the majority of students believe that their learning experiences have an effect on their motivation and learning strategies. The effect could vary -either positively or negatively -and could be felt either consciously or unconsciously (Sakiz, 2017). In relation to this study, we conceptualized students' belief as their attitude toward challenging learning strategies, especially in PBL. In general, students' behavioral response to projectbased instructions can be classified into two categories, namely 'challenge seekers' and 'challenge avoiders' (Meyer et al., 1997). For students who are challenge seekers, their learning goal as well as their threshold level for failure is high. Meanwhile, students who are challenge avoiders practice surface learning strategies, are focused on performance achievement and respond to failure negatively. Accordingly, it appears that students' beliefs will influence SE implementation. As a result, teachers must be aware of and be sensitive to the appropriateness of SE integration by considering factors such as students' educational level or grade level.
In terms of the implementation of integrated STEM, there are arguments regarding when and how integrated STEM should be taught to students. Some scholars argued whether students need to master the concepts of the disciplines before integrated STEM education can be implemented or if they can learn the concepts through integrated STEM teaching and learning (NAP, 2014;English & Kirshner, 2015;Thibaut et al., 2018). Since STEM is a combination of multiple disciplines, others questioned how mastery of the different disciplines are being carried out (NAP 2014). Apart from these issues, it is also imperative to know the effectiveness of integrated STEM toward student achievement as emphasized by English and Kirshner (2015) in his study. Therefore, this meta-analysis study was conducted to examine students' achievement in science subject through SE integration.
Becker and Park (2011) and Mustafa et al. (2016) investigated the effect of integrated STEM on students' achievement. Becker and Park (2011) examined the effects of an integrative approach based on subject integration where their preliminary meta-analysis data revealed that the 28 studies from 1989 to 2009 involving elementary, high school and college students indicated high effect (es=0.63). In other words, integrative approaches were found to have positive effects on the achievement of the students. The results showed that the integration of four STEM subjects exhibited the highest effect size (es=1.76). Meanwhile, Mustafa et al.'s (2016) study which focused on effective strategies for integrated STEM education revealed that project-based learning approach was deemed a dominant strategy in STEM education implementation. Even though meta-analysis study on STEM integration and its impact on achievement has been conducted, there is still a need to perform further review based on two reasons: (1) previous meta-analysis only covered studies from 2000 to 2009; and (2) effect size was not calculated. Thus, readers were unable to understand the magnitude of the differences in the previous study because the impact of the study's outcome was not determined. However, to our knowledge, up to the point of conducting this study, no metaanalysis study has ever focused on the impact of integrating science and engineering teaching approach on students' achievement.
In this meta-analysis study, we conceptualized students' achievement as understanding of a science subject. Tran and Nathan (2010) revealed that the limited integration of science into engineering courses has resulted in reduced achievement gains in science. To attain a better focus in this meta-analysis study, the integration of SE highlighted in this study was based on four (4) objectives. The first was embedding engineering practice into science subjects that influence student's achievement to understand the science concepts and enhancing scientific process skills. The second was to understand the most effective science subject in the SE integration. The third was to investigate the appropriate teaching strategies to enhance teachers' skills. Fourth and last objective was to identify the appropriate grade level to implement SE integration.

METHODS
This meta-analysis study was based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline by Moher et al. (2009). This guideline provides a series of stages as outlined in Figure 1, which starts with the process of identification, screening, eligibility, and inclusion. These are then followed by the coding process, extraction of effect size and interpretation of effect size  (Moher et al., 2009) For the process of identification, the electronic databases used to search relevant articles (including unpublished work) for this study included EBSCOhost, Emerald, Scopus, Web of Science and ERIC. Therefore, during the identification process, we practiced several steps such as: (i) used Boolean search "+, -, AND, NOT"; (ii) used specific keyword terms and their variants including integrated STEM, STEM education, student achievement, student outcomes, engineering design, Learning by Design, Design Based Science, Design Based Learning, problem-based learning, project-based learning, project-oriented quantitative problem-based learning, inquiry and problem solving, school; (iii) used alternative keywords for achievement such as ''performance, success, outcomes''; (iv) combined keywords on achievement with science disciplines such as "Biology, Chemistry, Physics, Science"; and (v) used references to track other suitable articles. After the initial identification process, the results yielded a total of 671 articles from Web of Science, 173 from Scopus, 266 from Emerald, 712 from EBS-COhost and 566 from ERIC. In the meantime, an additional 15 studies were identified following manual searches using the reference lists of the recovered articles to locate other related articles that might have been omitted during the database search. Hence, a total of 2403 articles were retrieved. However, after careful identification process, 77 duplicate studies were removed.
After the screening process, 2263 studies were excluded and only 63 studies were eligible. The 63 studies that went through the screening process were then filtered using a set of criteria. At this stage, we prioritized the articles that were: (1) quantitative research studies; (2) completed after the year 2000; (3) published in the English or the Malay language; and (4) comprised school samples. In addition, we also excluded studies which: (i) did not measure the relationship between integrated STEM (science and engineering) and students' achievement; and (ii) did not have enough data or basic statistics to calculate the effect size. The main rationale for these inclusion criteria was made based on the previous coverage of integrated STEM meta-analysis and because of other substantive reasons. At the end of the process, the number of studies qualified for inclusion was only 14.
The related articles were coded to describe the final analytic sample based on the context of the analysis. However, there were inherent challenges in categorizing the range of measurement approaches across the studies. Therefore, several steps were taken to maximize coding accuracy. First, before moving onto the included articles, a calibration exercise was conducted on the previous excluded articles which included reconciling differences and refining the current codebook. The codebook included all relevant information by adopting the MUTOS framework (Cronbach & Shapiro, 1982), namely U (grade level), T (teaching approach, teaching strategy, types of subject integration) and O (student achievement). Next, the effects of integrated STEM between the combinations of SE focusing on student achievement in all 4 science subjects (S-science, P-physics, C-chemistry, B-biology) were measured.
The effect size in this study was calculated using statistical tools from credible online sources (www.psychometrica.de/effect_size.html and www.campbellcollaboration.org/escalc/html/ EffectSizeCalculator-SMD1.php). Hattie's Continuum Effect (2009) was used to evaluate and explain the calculation of effect size (Cohen's d). This interpretation was chosen because it applies to specific educational contexts. Based on Hattie (2009), 0.2 is considered a small effect, while 0.4 is a medium effect and 0.6 a large effect. Thus, 0.4 was considered as a hinge point which was interpreted as a 'greater than average influence' on achievement. Additionally, based on Hattie et al. (2014), the factors on the left of the scale were those with decreased achievement while those on the right of the scale referred to increased achievement. Effect size below zero would not affect the results of the test.

RESULTS AND DISCUSSION
The impact of integrating SE teaching approach that focused on student achievement was measured. A total of 26 calculated effect sizes were obtained from 14 studies based on a sample size of 33,791 (Table 1). Positive effect was obtained with the overall mean effect size of (d=2.61, high effect). This suggests that SE integration has a major effect on students' achievement. This outcome is aligned with the findings of previous studies which stated that the integration of SE enhanced students' achievement in science subjects (Brown, 2017;Foster et al., 2013;Starr et al., 2020).
SE is required to solve many problems and to deepen students' understanding by applying scientific knowledge to solve engineering problems (Cunningham et al., 2020). The integration of SE indirectly results in students mastering math concepts and linking scientific concepts with technology (Asghar et al., 2012); it also enhances student's metacognition including creativity, critical thinking, problem-solving, and knowledge application (Goodpaster et al., 2012). Furthermore, it is also believed that student's interest toward science can also be improved through STEM integration (Dare et al., 2014;Bagiati & Evangelou, 2015;El-Deghaidy et al., 2017). In addition, integrating engineering design processes into science teaching through solving real world problems increases students' interest in STEM subjects and careers (Barth, 2013). Hattie et al. (2014) explained that even though meta-analysis is powerful because of the placement of all the effect size on a common continuum, the potential multidimensionality of the findings from the moderator variables needs to be considered too. Therefore, two moderator variables, namely T (teaching approach, teaching strategy, types of subject integration), and U (grade level) were also explored (refer to Table 1). Six teaching approaches were utilized to implement the integrated STEM lesson: (i) Integrated STEM-based (1, d=0.53); (ii) STEM Education-based Curriculum (1, d=0.80); (iii) Engineering Design Process (17, d=55.37); (iv) System Design (1, d=0.89); (v) Design-based Learning (2, d=3.73); and (vi) Design-based Science (4, d=6.77). The overall mean effect size results for the teaching approaches are shown in Figure 2.
Based on the overall mean effect size results presented in Figure 2, the highest calculated effect size was obtained from the Engineering Design Process (d=3.26, high effect) while the lowest was from the Integrated STEM-based (d=0.53, medium effect). SE based approach in this study can be interpreted as integrating engineering concepts and applications into various content areas in the curriculum (Cunningham & Carlsen, 2014). Based on the results, EDP yielded the highest calculated effect size, indicating that it is the most appropriate approach to integrate SE into the curriculum. In general, ED is used in multiple means in practice and the framework serves as a pedagogy, disciplinary practice, and a disciplinary core idea (Purzer, 2017). Therefore, ED serves as a structure in SE-based approach since they both have Figure 2. Overall Mean Effect Size for Teaching Approach parallel processes and similar problem-solving characteristics while also assisting teachers in teaching scientific inquiry (Cunningham & Carlsen, 2014). Moreover, SE based approach allows the implementation of PBL such as hands-on activities (Cunningham et al., 2020) and the application of scientific knowledge during problem-solving, which promotes new development of understanding among students (Barth, 2013). Dynamic learning approaches need to be tailored to a new curriculum in line with the students' specific capabilities, interests, and motivation toward STEM (Bagiati & Evangelouu, 2015;LaForce et al., 2016). Table 1 also describes teaching strategies that demonstrated positive effects on students' achievement. The overall mean effect size results for teaching strategies are shown in Figure 3.  Figure 3, the overall mean effect size for teaching strategy on student achievement estimated a large effect (d=7.83). There were seven types of strategies used to implement the integrated STEM (SE) lesson, namely: (i) Engineering Design Module (7, d=5.03); (ii) Engineering Curriculum (2, d=1.97); (iii) Engineering System Based Unit (3, d=0.66); (iv) Scaffolding Design Based Learning (1, d=2.09); (v) Design System Based Unit (3, d=0.66); (v) Engineering Design Based Science Unit (4, d=6.77); (vi) Pre-Engineering Project-Collaboration with Industry (1, d=1.33); and (vii) Engineering System Module (2, d=1.49). The effect size varied depending on the strategy used. The results indicated that the highest calculated effect size was obtained from the Engineering Design Science Unit.
Teaching strategy is a technique or method that is used to meet the learning objectives. A successful teaching approach would have an influence on students' academic achievements. Teachers therefore need to identify instructional goals and choose appropriate tasks by gauging students' capabilities (Householder & Hailey, 2012). Hence, the tasks must fit students' ability to increase their understanding and interest in STEM education (Honey et al., 2014;Han et al., 2015;Yaki et al., 2019); otherwise, the students may get discouraged and lose interest in learning (Householder & Hailey, 2012). Teachers' conventional perception that engineering integration is only suitable for high-level or talented students has been dismissed in a number of studies where it was demonstrated that regardless of the students' achievements, students performed relatively well when engineering concepts were incorporated in the lessons (Ozcan & Koya, 2019;Yaki et al., 2019). Thus, Engineering Design Unit strategy was identified as the most effective strategy to integrate with SE, with a mean effect size of 4.20 (high effect).
Meanwhile, Table 1 also revealed that students demonstrated positive response toward integration of SE in lessons. The overall mean effect size results for types of science subject in SE integration are shown in Figure 4.  Figure 4, effect size for types of subject integration was estimated to be a large effect (overall mean d=22.70). Meanwhile, for types of subject integration, all 4 science subjects (S, P, C, B) which were integrated with Engineering (E) were examined. However, none of the articles was suitable for calculating the effect size with regard to integration of Biology and Engineering (BE) subjects. Based on the graph in Figure 4, Science and Engineering (SE) indicated the highest value of effect size (21, d=65.3) and overall mean effect of 3.11 which indicated high effect, followed by Physics and Engineering (PE) (2, d=2.13), and Chemistry and Engineering (CE) (1, d=0.66). SE also yielded the highest value of effect size in Science subject (S) compared to P and C. The mean effect size of SE indicated a large effect. This finding concurs with Honey et al. (2014) and Walkington et al. (2011) who pointed out that engineering-based problems are oriented toward science practices. Moreover, Lachapelle and Cunningham (2014) mentioned that engineering education is science-oriented rather than mathematics oriented. Science is perceived as a real-life explanation, while engineering is accepted as a solution by humans (Bertoni, 2019). Therefore, integrating SE through EDP allows teachers to use inquiry learning and scientific reasoning to help bridge discipline boundaries between these two subjects (Barth, 2013). However, STEM integration will only succeed with qualified science teachers. The overall mean effect size results for grade level are shown in Figure 5. Based on Figure 5, all grade levels indicated a high effect. The overall mean effect size for all 3 grade le- Figure 5. Overall Mean Effect Size for Grade Level vels exhibited a large effect (d=22.70). However, compared to middle or high school, elementary school scored the highest effect size. The overall mean effect size for middle school was estimated at 2.22, while for elementary school, it was 8.37 and for high school, the effect size was 0.42.
It appears that among all three school levels, the elementary school children gained a better impact. One possible reason is that the middle and high school students are more geared to preparing themselves for examination. Meanwhile, elementary school students are free to explore and demonstrate their curiosity. Thus, there is a feasible need to start integrating STEM education at an early age (Ong et al., 2016;Soylu, 2016). Wyss et al. (2012) claimed that middle school is a critical time to enhance students' motivation and interest in STEM education. This statement is coherent with our findings which revealed that middle school demonstrated a large effect (mean effect size= 2.35). Therefore, adding EDP into the middle school curriculum would be an effective approach even for exam-oriented countries like Malaysia. The justification behind this is because STEM learning through ED increases middle school students' interest in STEM subjects and careers (Shahali et al., 2019). During middle school, students can master mathematic concepts through hands-on EDP activities. As for the high school setting, engineering elements in a design should reflect professional practice in engineering. Householder and Hailey (2012) provided some guidelines for implementing EDP at the high school level. Although EDP is new at the elementary school level, more effort is needed to successfully infuse EDP into their curricula (English & Kirshner, 2015;Cunningham et al., 2020).

CONCLUSION
Based on our findings, it is implicitly understood that not many studies have demonstrated statistically significant results for consideration in this meta-analysis study. In other words, the studies that investigated the effects of integration of SE on achievement may not exhibit the intended contribution. Thus, this meta-analysis study is timely as it can help to identify the forms of integration of SE that could provide significant effects on achievement. The findings demonstrated that integrating SE into the curriculum has a large effect on student achievement. Therefore, integrating SE will: (i) effectively improve knowledge in science; (ii) increase students' enrolment in science; and (iii) inspire and interest students to try engineering career paths. This study, however, did not review studies on students' diverse abilities. Students need different methodologies to help them become independent and strategic learners. More research is thus necessary to determine how science teachers could incorporate ED to suit the various abilities of students. The selection of teaching strategy is crucial where teachers must pay attention to choosing the most suitable approach for the betterment of the students' performance. Hence, future research could look into the most effective strategies in implementing integrated SE approach based on students' varied abilities.