Effects of Digital Science Teaching Materials on Conceptual Understanding and 21 st Century Skills: A Metaanalysis

: Innovative learning can improve students' 21st century skills. However, previous research states that students' 21st century skills such as critical thinking, creativity, collaboration and communication are still low. The solution to this problem is to analyze learning using electronic teaching materials. The aim of this research is to analyze the magnitude of the influence of a number of similar studies on students' 21st century skills. The research method can be classified as meta-analysis. In this meta-analysis method, the effect size of each similar study is determined and hypotheses are tested from a number of similar studies. In this research, 20 similar articles from national and international journals were analyzed. From the results of the analysis it can be stated that science teaching materials have a positive effect on 21st century skills based on the type of teaching materials, level of education and 21st century skills. Therefore, the use of science teaching materials is effective in improving students' 21st century skills both in terms of various types of teaching materials, level of education, and subjects.


Introduction
The 21 st century has brought about a shift in the skills that are necessary for success in the modern world.Here are some key findings from the search results on how 21 st century skills can encourage students' willingness to learn.A meta-analysis of studies on 21 st century skills identified 16 skills in three broad categories: foundational literacies, competencies, and character qualities (Khalid et al., 2015;Bredow et al., 2021).Foundational literacies, such as literacy and numeracy, serve as the base upon which students need to build more advanced and equally important competencies and character qualities.Acquisition of these skills has been the traditional focus of education around the world (Szymkowiak et al., 2021).A metaanalysis of 21 st century Four C's skills among K-12 students found that these skills, which include critical thinking, communication, collaboration, and creativity, are essential for success in the modern world and can be developed through education (Hanipah, 2023).Overall, the meta-analyses suggest that 21 st century skills can encourage students' willingness to learn by enhancing their motivation, engagement, and learning outcomes.Educators can incorporate these skills into their teaching practices to prepare students for success in the modern world.
The availability of appropriate facilities and infrastructure in the learning environment can significantly impact the learning process of students.Here are some ways in which the learning process can be supported by the facilities and infrastructure available in the learning environment.Creates a positive environment: The physical environment of the school speaks to the contribution that safe, clean, and comfortable surroundings make to a positive school climate in which students can learn.Physical environment is related to teachers' levels of absenteeism, effort, effectiveness in the classroom, morale, and job satisfaction (Dewi et al., 2021).Increases student focus: Studies have shown that thriving learning environments help increase student focus and retention of information.As students continue to grow and evolve as learners, so, too, do the environments in which they learn (Magdalena et al., 2020).Supports individualized learning: Learning environments can be designed to meet the individual needs of students, allowing teachers to continually optimize their learning environment to help students meet their educational goals.
The availability of technology has significantly impacted the learning process of students, providing them with more choices than ever before.Here are some ways in which technology has changed education.Increased access to education: Technology has made education more accessible, allowing students to learn anytime and anywhere (Haleem et al., 2022).With the widespread availability of smart devices that can connect to the internet, a new era of 'anytime, anywhere' education is on the horizon.Self-directed learning: Students can now learn many topics on their own by using internet resources and digital classrooms (Jeong, 2022).This has enabled self-directed learning, where students can take control of their own learning and pace themselves according to their needs (Magdalena et al., 2020.Unlimited access to knowledge: Digital technology provides unlimited access to knowledge, allowing students to benefit from many sources of inspiration (Qadir, 2023).Short educational clips: YouTube and other video-sharing platforms have made it possible for educators to create short educational clips that can be downloaded and integrated into their teaching.This has made it easier for students to learn complex concepts and has made learning more engaging.In summary, the availability of technology has significantly impacted the learning process of students by providing them with more choices, empowering learning, enabling selfdirected learning, providing unlimited access to knowledge, and making learning more engaging.
The use of technology in learning can significantly help students master their studies.Here are some ways in which technology can support student learning.Easy access to information: Technology provides students with easy access to information, which can accelerate learning and deepen their understanding of difficult concepts, particularly in STEM (Jesionkowska et al., 2020).It can also foster student engagement for auditory and visual learners (Rahmi, 2021).Self-directed learning: Technology enables self-directed learning, where students can take control of their own learning and pace themselves according to their needs (Kartikasari et al., 2021) In summary, the use of technology in learning can significantly help students master their studies by providing easy access to information, increasing engagement, enabling self-directed and personalized learning, improving comprehension, and making learning more fun.
The integration of technology in learning materials can significantly improve students' skills.Improved access to learning materials: Technology can provide students with easy access to learning materials, which can accelerate learning and deepen their understanding of difficult concepts (Asrizal & Utami, 2021).Increased engagement: Technology can increase student engagement by providing interactive and fun opportunities to practice what they learn (Zan, 2021).It can also foster student engagement for auditory and visual learners (Rahmi, 2021).Enhanced creativity: Technology can be used to enhance students' creativity by providing them with tools to create and design their own learning materials (Haleem et al., 2022) Improved collaboration: Technology can facilitate collaboration among students, allowing them to work together on projects and assignments (Qadir, 2023;Kusumastuti, 2020).In summary, the integration of technology in learning materials can significantly improve students' skills by providing easy access to learning materials, increasing engagement, enabling personalized learning, improving problem-solving skills, enhancing creativity, and improving collaboration.
Despite the potential benefits of technology integration in learning, many studies have shown that students still struggle with mastering concepts (Asriz & Utami , 2021).Limited student access: Students may not have access to technology outside of school, which can limit their ability to practice and apply what they learn (Haleem et al, 2022).Insufficient integration: Technology may not be integrated effectively into the learning process, which can limit its impact on student learning (Magdalena et al., 2020).In summary, while technology integration in learning has the potential to enhance student learning, there are several factors that can limit its effectiveness, including inadequate teacher training, lack of infrastructure, limited student access, insufficient integration, and resistance to change.
Based on the search results, there are several reasons why students may still struggle with mastering concepts despite the availability of learning resources and technology: Lack of social skills: A study found that students' lack of social skills can negatively impact their academic achievement (Rahayu, 2019).Insufficient 21 st century skills: Another study found that students' 21 st century skills, such as collaboration, problem-solving, and self-directed learning, were not adequately developed in some schools (Mu'Minah & Aripin, 2019).Field of study: Another study found that students in the natural sciences field of study had the lowest selfdirected learning skills scores, while students in the fine arts field of study had the highest scores (Haleem et al., 2022).Lack of early skills: Research has shown that children who enter school with specific early skills are more advantaged than their peers in terms of future academic achievement (Jeong, 2022).In summary, despite the availability of learning resources and technology, students may still struggle with mastering concepts due to factors such as lack of social skills, insufficient 21 st century skills, poor study skills and organization, gender differences, lack of early skills, and field of study.
The results of initial research were carried out by reviewing literature articles related to the influence of the use of digital teaching materials on students' mastery of 21 st century concepts and skills.Many publications discuss the influence of teaching materials on 21 st century skills.Digital teaching materials are able to encourage students' mastery of concepts (Musaddat et al., 2021).Digital teaching materials that are able to shape students' critical thinking abilities (Sagita & Syamsurizal, 2021).Several research results state that there is an influence of digital teaching materials on students' creative skills (Khamidah et al., 2019).Digital science teaching materials have a positive influence on 21 st century skills (Sari & Atmojo, 2021).However, this is different from research results which state that there is no effect of using digital science teaching materials on improving students' skills (Ramdani, 2020).The differences in the results of this study became a reference for researchers to conduct a meta-analysis of the influence of digital science teaching materials on students' mastery of 21 st century concepts and skills.
Several researchers have suggested using digital learning materials in science education to help improve students' mastery of concepts.Here are some examples of digital learning materials that have been proposed to enhance student learning, based on the search results: Digital comics: A study proposed the use of digital comics as a learning resource to help elementary school students master science concepts (Azamain et al., 2020).Interactive digital books: Interactive digital books have been proposed as a learning resource to help students improve their science literacy (Suprapto et al., 2022).These books can be accessed by students and teachers both online and offline, and they are free of charge, making them accessible to all students.Digital modules: A study proposed the development of digital modules based on flipbooks to support online learning in science education for fourth-grade students (Fitrianawati & Setiyawati, 2021).The digital modules were found to be effective in motivating students to learn independently (Serevina et al., 2018).In summary, digital learning materials such as digital comics, interactive digital books, mock-ups, and digital modules have been proposed as solutions to help improve students' mastery of science concepts.These materials can be accessed online and offline, are free of charge, and can be used to support independent learning.
The solutions of several previous researchers have several limitations.The limitation of previous researchers was only discussing one teaching material (Aulia et al., 2021;Kusumawati, 2020).There is no clear measure of the relationship between teaching materials and students' 21 st century skills.Using only one teaching material, also only focusing on one skill.Based on these problems, a meta-analysis was carried out on the influence of digital science teaching materials on students' mastery of 21 st century concepts and skills.This research with effect size analysis aims to: 1) analyze the influence of using digital science teaching materials in terms of the type of teaching materials, 2) analyze the influence of using digital science teaching materials on 21 st century skills in terms of variations in education levels.3) analyze the influence of digital science teaching materials in terms of subjects, 4) analyze the influence of using digital teaching materials on students' mastery of concepts and 21 st century skills.
A comprehensive study regarding the influence of science teaching materials needs to be carried out.Science digital teaching materials apply active students in learning, students can use teaching materials anywhere and anytime.With the increasingly rapid development of technology, it will be a loss if it is not used in learning.The use of technology in learning is using digital teaching materials that utilize digital technology.Thus, it is necessary to carry out a metaanalysis of the influence of science teaching materials on increasing students' mastery of 21 st century concepts and skills.

Method
The research method in the research is meta analysis.Meta-analysis is a type of systematic review research (Ridwan, 2021).Analyzing findings from previous scientific publications is known as a systematic review.This systematic review research found and analyzed effect size values from previous publications.The data found in the article can be said to be quantitative data.Meta-analysis research on the influence of digital science teaching materials on increasing mastery of 21 st century concepts and skills.
At the data collection stage from the Google Scholar database, 2001 search results were obtained and then 20 articles were selected that met the criteria.The selection of articles used used selection based on inclusion and exclusion criteria.The criteria used in selecting articles for this meta-analysis were taken from national and international journals.Inclusion criteria in this research include: search keywords, namely "digital teaching materials", "concept mastery", "science teaching materials" and "critical thinking skills" "creative thinking skills".Articles published between 2014-2023.Articles in Indonesian or English.The selection of articles was in accordance with the exclusion criteria, namely: having statistics on the number of students.The selection of articles was in have the average score of the experimental class and control class, or the pretest posttest score of the control class and experimental class.The selection of articles was in teaching materials used in the scope of science.
Meta-analysis of the influence of digital science teaching materials on mastery of 21 st century concepts and skills.Important research variables influence research results.Meta-analysis research variables use moderator variables.The moderator variables used are type of teaching material, subject, level of education, and type of skill.To see the influence of each moderator variable, a meta-analysis was carried out on the articles to obtain 20 articles.
This research obtained data by conducting a metaanalysis of articles from journals.The meta-analytic nature is to carry out comparative descriptive analysis.Analyze the results of the article data and make comparisons with other articles then state conclusions.At the data collection stage from the Google Scholar database, 2001 search results were obtained, then 20 articles were selected that met the criteria.The data analysis techniques (Moher et al, 2009) used in this research are identification, filtering, eligibility and inclusion.The research found is in the form of quantitative data so that the data in the article can be analyzed.Quantitative research states the analysis of data that states the relationship between variables in the article.Determining the relationship between variables can be determined using the effect size.The steps for tabulating effect size data are: identifying research variables, identifying statistical data in the article, calculating the effect size using the effect size formula.The results of the meta-analysis are expressed in effect size values.Results of data analysis using Ms. Excel and JASP software to calculate effect size statistical data.The effect size value obtained will be categorized based on Cohen's standards (Cohen, 2018).Then the Q parameter value is obtained as a determinant that the data obtained is heterogeneous.If the values are heterogeneous, then proceed with hypothesis testing using random effects and it can be seen that the pvalue is small from the alpha value = 0.05.The classification of effect size value categories is shown in Table 1.

Characteristics of meta-analytic studies
The research results were obtained by collecting 20 articles from various journal sources.Journal sources are taken from national and international journals.Articles collected via Google scholar and Eric.The 20 articles obtained were articles related to digital science teaching materials for mastering 21 st century concepts and skills.The 21 st century skills in question are critical, creative, communicative and collaborative thinking skills.These articles will be grouped based on ariable moderators, namely class level, subject, type of teaching material, and 21 st century skills.The data that will be analyzed is quantitative effect size data from 20 articles which can be seen in Table 2. Based on the data in table 2, digital science teaching materials in the period 2014 to 2023 are most common in 2022.The use of digital teaching materials has increased from 2019 to the present.The use of digital teaching materials in learning can improve students' 21 st century skills (Puspitasari, 2019;Suharyat et al., 2022).The lowest effect size value of -1.37 shows that there is no influence of teaching materials on improving 21 st century skills.The highest effect size value of 1.74 is in the very high category.The average effect size value obtained was 0.82 in the high category.The results of this research are relevant to previous research regarding the use of digital teaching materials.The effect size obtained was 0.68 in the high category (Saregar et al., 2016).Other research found an effect size related to digital teaching materials of 0.691 in the high category (Risnawati et al., 2022).The difference in effect size results is influenced by several other factors.

Heterogeneity Test Results
The research results obtained effect size data for each article and a heterogeneity test was carried out.The heterogeneity test will obtain data on the Q, P and I values.The heterogeneity value can be seen from the Q and I values.For the P value, the hypothesis is accepted if the p value <0.05.Then it is stated that there is an influence of the use of integrated teaching materials with the PBL-BL model on 21 st century skills.Heterogeneous data on the influence of the use of integrated learning models of teaching materials on 21 st century skills are presented in Table 3.The results of the heterogeneity test show that the effect size of the 20 articles analyzed is heterogeneous.Q = 329.07;P<0.001, 95% confidence level.With the heterogeneity test results obtained, the random effect model is suitable for estimating the average effect size value (Zan, 2023).Random effect results show that there is a significant influence of the use of digital teaching materials on 21 st century skills.The positive influence of the use of digital teaching materials on 21 st century skills has a value of 0.82 in the very high category.The use of teaching materials can improve 21 st century skills in line with previous research which obtained an effect size value of 1.71 in the very high category (Suharyat et al., 2022).
The effect size values obtained vary due to the presence of moderator variables in the research.The moderator variables in this research are variations in types of teaching materials, learning models, education levels, and skills.The large variety of moderators will influence the research results.To determine the effect of integrated teaching materials on learning models based on moderator variables seen from the combined value of heterogeneity (QW) and medium heterogeneity (QB).

The Influence of Science Teaching Materials Based on the Type of Teaching Materials
The first moderator variable is seen from the diversity of teaching materials contained in the article.Meta-analysis was carried out to see the influence of the type of teaching materials on 21 st century skills.The results of the meta-analysis based on teaching materials can be seen in Table 4.
Based on the analysis results in table 4, the results of heterogeneity and p-value are obtained.The Q value of 124.68 states that the moderator variable has heterogeneous data because it is greater than the total df value.The highest effect size value was obtained for emodule teaching materials, namely 1.02, which is included in the high category.There is a big influence in the use of e-module teaching materials on improving students' 21 st century skills.The lowest effect size value is found in e-book teaching materials, namely 0.34, which is included in the low category.So the influence of e-books on improving 21st century skills is only small and insignificant.The p-value of the moderator variable is less than 0.05 so the H0 value is rejected.The research results show that there is a significant influence of the use of teaching materials on 21 st century skills.This is in line with previous research which states that the use of digital science teaching materials can help improve 21 st century skills (Jannah & Atmojo, 2022;Yunita & Hamdi, 2019).

Influence of Science Teaching Materials Based on Education Level
The third meta-analysis of the moderating variable for science teaching materials is education level.The use of digital teaching materials for 21 st century skills is also influenced by the level of education.The results of the meta analysis based on education level can be presented in table 5.
Senior High School 12 0.88 0.60 1.18 The results of data analysis obtained a Q value of 38.39 which was greater than the df value.So it can be said that the data used is heterogeneous.Then the random effect will be tested and a small p value is obtained, namely 0.05 so that H0 is rejected.The results of the analysis show that there is a significant influence of the use of teaching materials on 21 st century skills.The highest effect size value is at the high school level with a value of 0.88 in the high category.The use of digital teaching materials is suitable for use at the high school level in improving 21 st century skills.The lowest effect size value is at the elementary school level, namely 0.67.The use of teaching materials has an influence on improving 21 st century skills at the elementary school level.This is in line with research which states that the use of teaching materials can improve students' 21 st century skills ( Nazifah & Asrizal, 2022;Asrizal et al., 2022).

The Influence of Science Teaching Materials Based on the Components of Mastery of 21 st Century Concepts and Skills
The fourth moderator variable analyzes the influence of using integrated PBL and BL model teaching materials in terms of 21 st century skills.21 st century skills are critical thinking, creative, collaboration and communication skills (Khoiri et al., 2021 ).Based on the 20 articles analyzed, only two skills were found, namely critical and creative thinking skills.The results of the skills-based meta analysis can be presented in table 6.Based on the results of the analysis in table 6, the heterogeneity value and p-value can be stated.The Q value shows 20.29 which is greater than the df value.It can be seen that the Q value shows that the data obtained is heterogeneous.Next, a random effect hypothesis test was carried out and a p value <0.05 was obtained.This shows that there is an influence of the use of teaching materials on each 21 st century skill.The highest effect size value is found in critical skills, namely 0.96 which is in the high category.The results of this meta-analysis research reveal the magnitude of the influence on improving critical thinking skills (Ramdani & Susilo, 2022;Yunita & Hamdi, 2019).The smallest effect size value for concept mastery is 0.68 in the medium category.The results of the research state that there is an influence of the use of digital teaching materials on concept mastery.In line with research results, the use of digital science teaching materials improves creative thinking skills (Hasibuan et al., 2022;Hursen, 2021).

Conclusion
Based on research, the general conclusion is that meta-analysis carried out on digital science teaching materials is able to improve students' mastery of 21st century concepts and skills.Furthermore, based on the objectives, 4 conclusions were obtained.First, the influence of the use of digital science teaching materials on the diversity of teaching materials is greatest in emodules and smallest in e-books.Second, the influence of using digital teaching materials is greatest in high school and smallest in elementary school.Third, the influence of the integrated learning model is greatest on critical thinking skills and smallest on concept mastery.

Table 1 .
Classification of effect sizes

Table 2 .
Characteristics of meta-analytic studies

Table 3 .
Meta analysis results

Table 4 .
Effect of science teaching materials according to type of teaching material

Table 5 .
Influence of science teaching materials according to education level

Table 6 .
Effect of science teaching materials based on 21 st century skills components