DEVELOPING A CONCEPTUAL MODEL OF LEARNING ANALYTICS IN SERIOUS GAMES FOR STEM EDUCATION

Utilizing serious games as teaching aid can stimulate students’ interest in learning and enhancing students’ understanding of STEM learning. The use of serious game on learning analytics for STEM learning provides assessment data to measure students’ performances and achievements of predefined learning outcomes. While numerous researches on learning analytic in serious games have been conducted, studies on their association with STEM learning are scarce. Past studies also indicate that teachers use serious games in teaching but yet to utilize serious games as assessment tools. This situation leads to the need of developing a conceptual model of learning analytics in serious games for STEM education (APPS-STEM). The conceptual model was initially developed through the focus group approach and literature review to examine the learning metrics of serious games, which is appropriate for STEM learning before being verified by experts. Based on the result from expert review, the conceptual model of APPS-STEM contains nine themes, namely effectiveness, problem-solving thinking and creativity, flexibility, key stakeholders, emotions, serious game design, curriculum profiles, learning profiles, and target user norms. This conceptual model is expected to serve as a guide for stakeholders to implement learning analytics in serious games design for STEM learning. Further study will be the development of serious game prototypes to determine the effectiveness of the APPS-STEM model in the STEM learning paradigm. © 2020 Science Education Study Program FMIPA UNNES Semarang


INTRODUCTION
The concept of STEM education is an effort to engage students in learning science, technology, engineering, and mathematics that also has been recognized as one of the most important learning experiences in schools all around the world (Johnson et al., 2014). The combination of all four STEM disciplines into a single discipline is considered more practical and realistic to enhance the students' knowledge, skills, attitudes, and interest in Science and Mathematics subjects (Adnan et al., 2017).
STEM learning requires an active learning mode for students to apply STEM concepts through fun and challenging activities. According to Wang (2013), studies on STEM learning also represents high empirical efforts to build a better understanding of the underlying factors that influence students' success throughout the STEM learning process. Besides, STEM learning is also a learning process that moves beyond facts and procedures by introducing concepts and engaging students in STEM practices such as developing evidence-based explanations, self-exploration of the visual programming language (Ismail et al., 2016), and students` critical thinking skill through project-based learning (Mutakinati et al., 2018).
Today, STEM education has become a priority in Malaysia's education line, conforming with the country's desire to increase the percentage of students' interest in STEM education, both at school and higher education level, thus enhancing the country's success and goals to boost the country's competitiveness, especially in the field of STEM globally. In Malaysia's context, STEM refers to the educational policy and choice of the school curriculum to enhance the competitiveness of science and technology to students. As embodied in the Malaysia Education Blueprint 2013-2025, the emphasis of STEM education at the school level is through curriculum and co-curricular activities with the support of various stakeholders (Malaysia, 2013). In order to gain students' interest in learning STEM-related subjects, teachers need to be smart in applying technology support resources to diversify teaching and learning methods and familiarize themselves with STEM projects and read about STEM to build or enhance accurate knowledge and understanding of STEM (Adnan et al., 2017).
Serious games are defined as games that are designed and used for purposes other than entertainment (Djaouti et al., 2011;Zyda, 2005). Breuer & Bente (2010) also argues that serious games are not just any commercial game intended solely to entertain or satisfy players. According to Hussaan & Sehaba (2013), serious game can also be defined as an intellectual challenge with specific rules laid down on a digital device. Serious games are designed to entertain, engage and develop new knowledge and skills (Eichenberg & Schott, 2017), provide meaningful content, titles, narratives, rules, and objectives that will achieve specific learning outcomes (Mitgutsch & Alvarado, 2012). The production of an effective serious game is highly dependent on the design needs of the user for each design component. Well designed serious games can provide powerful interactive experiences that can foster a player's learning, skill-building, and healthy development (Lieberman et al., 2009).
Serious games have emerged as an interesting new educational tool for learning STEM fields. It allows students to understand more practically than just learning the theory in class without practicing it in the real world. Serious games in education also have been reported to be effective in facilitating learners' holistic understanding of scientific conceptions, gaining cognitive skills, increasing the positive impact of learning and improving science teaching, providing flexible learning, improving learning outcomes, and improve cross-cultural communication competence (Zhonggen, 2019). Moreover, learning through technological tools seems to stimulate the interest of today's students as they are part of the Digital Native generation and are already familiar with technology in their daily lives (Savazzi et al., 2018).
According to Wu & Anderson (2015), the use of serious games makes STEM education more efficient. Serious games can attract students through challenging, fun STEM education activities (Cheng et al., 2015). Serious games are also used to gain students' interest and improve their understanding of STEM learning by using a tangible serious game to allow children to have a multi-sensory experience of concepts and facts that are difficult to learn with standard learning tools (Berta et al., 2016). Besides, the implementation of serious games to learn science subjects improving the students' creative thinking skills and motivation to learn (Putra & Iqbal, 2016).
In addition, a study by McLaren et al. (2017) found that students who use serious games in mathematics subjects enjoy more learning experience than students who do not use serious games. They also found that students had a positive feeling and were more confident about the subject when they used serious games and made fewer mistakes during the learning process. Meanwhile, studies by Ameerbakhsh et al. (2016) and Ceberio et al. (2016) found that the use of interactive simulations as one of the serious game genre, in ecological learning and physics has increased effectiveness, allowed learners to interact and engage in simulation more than non-interactive simulation, and helped students learn efficiently and enjoyably. These show that serious games are very suitable for use as a teaching tool in STEM education.
Currently, serious game studies focus on serious game analytics, specifically on assessment and learners' behaviors (Alonso-Fernández et al., 2019). Serious game analytics is defined as a metric that can be implemented through problem definition in training or learning scenarios using statistical models, metrics, skill analysis, and performance improvement and assessment in the use of serious games as essential training tools (Loh et al., 2015). Through the use of learning analytic in serious games, assessment data can provide the information needed to measure the effectiveness of serious games to achieve a set of learning objectives, as well as provide detailed information on changes that can be made to improve a decision. The learning analytic elements in serious games are employed to collect, measure, analyze and generate data about students and their contexts to understand and optimize learning and the environment (Siemens et al., 2013;Siemens & Long, 2011). The field of learning analysis relates to gathering, analyzing, and visualizing vast volumes of educational-related data. Learning analytics also leverages the capacity of big data and data-mining techniques to improve learning assessment (Serrano-Laguna et al., 2013).
User profile data and user behavior generated in the game environment are vital as they serve as evidence for problem-solving strategies and explain decision-making processes during the learning and training process. Additionally, the use of serious game analytics has undoubtedly helped to refine the design of the game, as well as validate its effects on players (Calvo-Morata et al., 2018;Cano et al., 2018) and to predict learning outcomes for players (Serrano-Laguna et al., 2018).
According to Loh et al. (2015), serious games establish a different major goal, which is to improve the players' skills and performance through training and instructions. Players who practice and learn through serious games will be able to play the game. Towards the end of the session, players will also be able to improve their skills and performance as well as acquire particular information and knowledge. Study by Daoudi et al. (2017) found that learners' assessment and measurement in serious games are extremely important for improving learning outcomes and maintaining their motivation. Besides that, learners' assessment and measurement are also relevant for exploiting to provide a solution that fits the profile and objectives or learning needs of the students.
Game usage patterns can contribute to the understanding of learning models that can later be used to predict the development of students' knowledge through performance, engagement, and resource processing sequences. Although getting access and the use of in-game data is challenging, with the advent of the latest technologies and emerging areas of learning analytics, researchers have the opportunity to study the use of an immense amount of dynamic data in serious games (Liu et al., 2017). Acquisition of user data generated in the game to help make decisions on game design and development has also become common among game developers. Besides, serious game developers and researchers are encouraged to engage with different types of data, as these data are needed to clarify issues related to measuring, evaluating, and improving performance in game-based learning and training (Loh et al., 2015). To assist certain parties in the development of serious games, Ifenthaler & Widanapathirana (2014) have developed a framework of learning analytic for the design, implementation, and evaluation of serious games for inclusive learning and teaching and Nguyen et al. (2018) have produced a systematic framework for integrating learning analysis into serious games for people with intellectual disabilities.
In the context of learning analytics in Malaysia, studies (Kumar & Hamid, 2017;West et al., 2018) show that academics have a high interest in the idea of using learning analytics for learning and teaching and its importance for student achievement in higher learning institutions. Although the use of serious games in education and particular STEM learning is growing, the use for the context of Malaysian students and teachers and the integration of learning analytics for STEM learning is still lacking. Teachers have begun implementing serious games as part of their teaching aids; however, they are found still unready to employ the games as assessment tools and would instead rely more on traditional methods such as written tests (Serrano-Laguna et al., 2012). Some of the main reasons why teachers are not inclined to use serious games as assessment tools are that they are unable to monitor assessments accurately, that games are less flexible and unchangeable after development, and that there is also a lack of detailed reports on games played, such as student interaction with games (Chaudy & Connolly, 2019).
Based on the stated issues and the importance of learning analytic in serious games, it indicates the need to conduct studies that focus on the in-game learning analytics for STEM education. Thus, the main focus of this study was to develop a conceptual model for learning analytics in serious games to improve and enhance STEM education (APPS-STEM) -the acronym is derived from Malay phases "Analitis Pembelajaran Dalam Permainan Serius Bagi Pembelajaran STEM"). The conceptual model of APPS-STEM can be used as a guideline for the implementation of learning analytics in serious game development for STEM education.

METHODS
The method used to develop the conceptual model of APPS-STEM involved two phases, namely analysis and design, and evaluation, as shown in Figure 1.

Figure 1. Method of Conceptual Model Development
Triangulation methods were used in the analysis and design phases, adapted from Zaki et al. (2017), but this study carried out focus groups instead of interviews. In addition, literature studies have also been performed to establish initial conceptual models. The expert review method adapted from Abubakar et al. (2016) is then used to verify the conceptual model.

Analysis and Design Phase
The study began by examining the analysis of learning analytic in serious games through conducting focus group interviews with students, experienced teachers, educational technology experts, and game developers that are appropriate to their needs and relevant in the context of STEM learning. A focus group is a group of individuals typically consisting of six to eight people who discuss a topic related to a theme that is moderated by moderators (Krueger, 2014). Therefore, a total of 14 respondents were selected and divided into two focus groups as shown in Figure 2. The selection of respondents was performed randomly to match the domain of this study. The focus group interview session was conducted by a moderator and an assistant who wrote down the discussion content (among the researchers) at Smart Lab, Faculty of Art, Computing, and Creative Industry, Sultan Idris Education University. The discussion time for both focus groups took 90 minutes, and the discussions were also recorded in the form of video and voice recordings.
The affinity diagram method was used to obtain learning analytic metrics. Through this method, a large amount of data (ideas, views, issues, etc.) was collected and compiled based on the relationships between them and were commonly used within a group of individuals who generate ideas for a discussion (Nancy, 2005).
In the context of this study, this method was carried out at the end of the discussion session where each respondent was required to provide some learning analytics metrics that they thought would be appropriate in serious games for STEM learning after they had acquired an image or understanding of the discussion related to learning analytics.
This affinity process was implemented using several sheets of sticky notes given to each member of the focus group. Respondents were required to write their ideas regarding learning analytic metrics on the given sheet and paste them on a whiteboard. Once completed, each of these metrics will be ranked on the basis of its priorities and relevance to the study and coded according to specific themes as shown in Figure 3. In addition to the focus group, literature reviews from established journals focusing on serious game analytics, learning analytics, and ingame metrics were conducted. The comparison and coordinating process between the results list of learning analytic metrics in the serious game from focus groups and the literature review were then carried out to design an initial conceptual model of APPS-STEM.

Evaluation Phase
The evaluation of the initial conceptual model of APPS-STEM will be carried out through an expert review to validate the identified learning analytic metrics in a serious game. Experts' reviews are critical to explain the validity of the instrument content by examining the scale and come to an agreement that the items contained in the scale represent items related to the concept being measured. According to Grant & Davis (1997), the number of expert panelists depends on the level of expertise required and the variety of knowledge itself. Some recent researchers have established their own set of experts, consulting at least five experts (Burns & Grove, 1993) and proposed between five to seven experts to evaluate the scale of domain content using a rating scale. In this study, six experts were consulted consisting of STEM teachers, game developers, and academics from the field of serious game and educational technology selected through purposive sampling techniques. Their demographic is demonstrated in Table 1. The instrument for the expert review includes expert personal details, learning analytic metrics of serious games and comments, or suggestions. Experts need to evaluate the metrics set out in the instrument based on three Likert scales, namely, agreed, neutral, and not agreed. The data collection lasted approximately four weeks, and the data collected for the review was then analyzed using the descriptive analysis to determine the percentages for each scale used.

RESULTS AND DISCUSSION
Based on the discussion results from both focus groups, the learning analytic metric items of STEM learning through serious games were coded into 11 themes such as norms, emotions, effects, evaluations, feedbacks, problem solving and creative thinking, time, flexibility, scoring, interface, and specialization. Meanwhile, five learning analytic metrics have been identified from relevant literature studies (Alonso-Fernandez et al., 2017;Alonso-Fernández et al., 2019;Ifenthaler & Widanapathirana, 2014;Kiili et al., 2018) as shown in Table 2. The results of the comparison and coordination process between the list of learning analytic metrics in serious games from focus groups and literature review yielded nine themes namely target user norms, learning profiles, emotions, serious game design, effectiveness, problem-solving and creative thinking, flexibility, curriculum profile, and key stakeholders. Based on these nine themes, initial conceptual models of the APPS-STEM model are generated.
An instrument containing 67 items of ingame metrics was developed to represent the nine themes. The results of the instrument review by six experts were analyzed as in Figure 4. The instrument validation results indicate that five out of nine themes received a significant percentage of approval from experts by more than 80%. The themes represented were emotions, effectiveness, creative and problem-solving thinking, flexibility, and key stakeholders. While for the rest of the themes there is an average percentage, between 60-79%, of agreement. Thus, the overall findings indicate that experts are satisfied with all the learning analytic themes for serious games for STEM learning provided in the instrument. Nevertheless, for some items that represent a particular theme with a 50% disagreement and a neutral percentage, the items have been further reviewed based on experts' comments to determine its importance to the themes.
Based on the assessment and feedback from the expert review, a validated APPS-STEM conceptual model is established as seen in Figure  5. According to Figure 5, there are nine main themes for learning analytics in serious games for STEM learning. The description of each theme is as follows; Effectiveness is the effective use of serious games to achieve predefined learning objectives depends on the accuracy of the facts as well as the continuous use of the game. The facts provided must be from authentic sources so that the reliability of the game content is excellent and teachers and students will not hesitate to practice it. It can be done by referring to the STEM curriculum which has been carefully curated by the STEM curriculum experts (Malaysia, 2013).
Problem-solving and creative thinking is about solving problems or identifying opportunities when playing a serious game. Among the learning analytic metrics identified for this theme, games should be designed based on problemsolving steps and based on the content of the subject. Besides, questions in the game should be tailored to Bloom's Revised Taxonomyleveland encourage students to be creative in delivering answers, thus improving students' creative thinking skills when learning STEM learning (Putra & Iqbal, 2016).
Flexibility-games should be accessible everywhere and not limited to schools only. Through flexibility, players will be able to easily return to previous game sessions, and players can learn or revise their previous lesson before attempting to play the game again. Zhonggen (2019) also found that serious games provide flexible learning by offering learners the opportunity to choose the time and place they feel comfortable to learn, without restricting themselves to fixed schedules and places.
Key stakeholders are stakeholders involved in the games such as students or players, teachers, parents, and designers or game developers (Alonso-Fernandez et al., 2017;Alonso-Fernández et al., 2019). For students, there is a need for informative feedback on the performance of their game, for example, their final score, comparing scores with other players, comparing their position or game history. While for teachers, they need general statistics such as how many students finish each level of the game, how long it takes to solve, or the most common mistakes students make. Parents can acquire real-time information about their children's learning, and game designers or developers can identify areas where many mistakes have been made that could potentially be improved from aspects of game design. The roles and benefits of each of the parties involved can also be found in the Malaysia Education Blueprint 2013-2025(Malaysia, 2013. Emotions are related to the players› feelings experienced during and after the game. Emotions can influence the learning analytic, students who enjoy learning are less stressed and faceless pressure of assessment. Self-confidence can also be assessed through speed and accuracy when playing. In addition, feeling entertained; interested (players will typically stay throughout the game if they are interested in learning or discovering something) (Berta et al., 2016), effort (more notable effort presents them to being a high-performance user) (McLaren et al., 2017), and satisfaction (satisfied users will come back to play).
Serious game design-Learning analytic metrics can be implemented through optimal game design as each game developed should possess stages of difficulty in order not to lose players' interest. Players are also allowed to go back and replay the easier levels even after advancing to the difficult stages to reduce error rates and improve the learning rate (Alonso-Fernández et al., 2019). In addition, designing games that encourage student engagement by adapting to challenges balanced with rewards will cause players to crave additional challenges for more good rewards. Therefore, motivational elements must additionally be included. Game design should also include real-time, past and future feedback, visualize the navigation path between displays to reflect player priorities and learning styles, whether it is linear or global (Ifenthaler & Widanapathirana, 2014), a design with an attractive and cheerful interface, containing animations, option to freely choose names and avatars that are representative of the game world and with performance levels.
Curriculum profiles are about course information, learning outcomes, teaching, and assessment materials. Relevant learning analytics metrics are according to students individually or by class performance (Examples: sessions, questions resolved, number of errors, accurate answer, and time ratios), in-game score distribution, distribution of answered questions by students, accuracy estimation, comparison of accuracy, and order of magnitude, shows the percentage of the tasks completed correctly, the effective playtime, the amount of time that a player takes to complete all the tasks, the highest level of play, the number of games played and the star rating that the player has or can earn (Alonso-Fernandez et al., 2017;Kiili et al., 2018).
Learning profile is the metadata about activities carried out in the learning environment to evaluate learning and predict performance (Alonso-Fernandez et al., 2017;Alonso-Fernández et al., 2019). Among the learning analytic metrics that can be used is the amount of time used, learning status, completion rate, rating, grade, player exploration strategy, characteristics of players› failure, number of sessions, speed, accuracy, and scores.
Target user norms are player demographics such as student background including gender, age, education level, and academic performance history are recorded. Alonso-Fernández et al.
(2019) also state the importance of understanding students› age and gender characteristics, as students with different learning characteristics will have different learning behaviors. The game must also be developed based on STEM learning that is appropriate for students in Malaysia whose primary language is used in the learning process or developed bilingually which is in Malay and English. Additionally, players can be grouped into a performance group based on in-game actions and in-game choices that are also relevant to prior knowledge.
All nine in-game learning analytics discussed above are expected to provide the assessment data needed to measure student achievement as well as the achievement of predefined learning outcomes in STEM learning as learning analytics involves the use of big data and data mining to boost learning assessment (Serrano-Laguna et al., 2013). Through the use of analytic learning in serious games for STEM learning, teachers will be able to take proactive steps to assist the lower performing students can also keep students motivated (Daoudi et al., 2017). Besides, information derived from student interactions with serious games can significantly improve the evaluation of games and allow both teachers and institutions to make evidence-based decisions and increase their attractiveness and confidence in the use of serious games in formal education (Alonso-Fernández et al., 2019).
Although there are previous studies on the learning analytics framework (Ifenthaler & Widanapathirana, 2014;Nguyen et al., 2018), they are not dedicated to the use of STEM education. Hence the proposed conceptual model of APPS-STEM will meet the needs of both students and teachers for STEM learning. This conceptual model is also useful as a guide for different parties, in particular, game developers who need analytic learning information that is appropriate for STEM learning using serious games. The use of standard learning analytic for target users will be able to benefit them. Students and teachers involved in STEM learning will therefore benefit if the serious game design takes into account learning analytics as suggested.

CONCLUSION
Teachers are able to diversify their teaching methods to provide a learning environment that is interesting to students and challenging by incorporating serious play elements into STEM learning. By incorporating the learning analytic into a serious game, teachers can easily collect, measure, analyze, and produce data reports that assist teachers to achieve their assigned learning objectives. Through the development of the APPS-STEM conceptual model, it is expected that the model will benefit indirectly in enhancing STEM learning in Malaysia. It could then be a catalyst for an educational transformation to bring about changes in STEM learning and teaching to help the country produce more skilled STEM-based workforce. However, the effectiveness of adopting this model in STEM learning also needs to be noted. Thus, the following study will focus on the development of a serious game prototype and will test its effectiveness on real users. ACKNOWLEDGMENTS This paper is based on the research project entitled "Model Analitis Pembelajaran Dalam Permainan Serius Untuk Pembelajaran STEM". The authors would like to extend their gratitude to the Research Management and Innovation Centre (RMIC), Sultan Idris Education University (UPSI) for the University Research Grants (code: 2018-0244-107-01) that helped fund the research.