The Effect of STEM-Based Mathematics and Natural Science Teaching Materials on Students' Critical and Creative Thinking Skills: A Meta-Analysis

: Students must possess 21st century abilities, including the ability to think critically and creatively, in order for education to be developed for this century. Teaching materials are required to support students' learning as they develop these skills. The development of the 21st century cannot be separated from technological developments which are developing rapidly so that Science, Technology, Engineering and Mathematics (STEM) based teaching materials are very popular to use. The goal of the study was to ascertain how STEM-based instructional materials affected students' capacity for critical and creative thought. The research method used is meta-analysis of 22 national and international journal articles. The research results obtained are with variations in the level of educational units, teaching material subjects, critical and creative thinking skills as well as the variety of teaching materials used state that the use of STEM-based teaching materials has a significant influence on students' critical and creative thinking skills. The conclusion from this meta-analysis research is that there is a significant influence from the use of STEM-based teaching materials on students' critical and creative thinking skills.


Introduction
The 21st century is known as the era of rapid development of knowledge, technology and talent. Formation of superior human resources is the main goal to be achieved in the 21st century. Education is very important in the 21st century to ensure students are able to learn and innovate, utilize technology and information, work together with others, and utilize life skills in the 21st century (Mukaromah et al., 2022;Nazifah et al., 2022;Pratiwi et al., 2019). The ability to think critically, think creatively and solve problems, communicate and collaborate, be creative and update, contextual learning, as well as information and media literacy skills are among the 21st century skills emphasized in the curriculum.
The needs and problems of the twenty-first century are impacted by changes in Indonesian learning habits. This can be seen from the progress of the education unit level curriculum which was refined into the 2013 curriculum and refined again into the 2013 curriculum revision and now has been refined again to become an independent curriculum. This is in accordance assertion with Festiyed (2014) that efforts to improve educational standards are always carried out through curriculum revisions or procurement of new textbooks. Learning science and physics requires supporting learning resources, such as teaching materials, in order to improve student learning outcomes and competencies. According to Desnita et al. (2018) learning media is a tool used to teach students about a subject and make it easier for them to understand it. 55 The existing reality has not met the desired expectations. According to research previously conducted by Ranti et al. (2019), half of the students had not reached the Minimum Completeness Criteria (MCC) and had not shown a thorough understanding of the material. In addition, textbooks and worksheets from publishers are still used which have not fully met the criteria, and the explanation of the material presented in the worksheets is still unclear in factual, conceptual and procedural detail. Lack of student enthusiasm in improving learning is a further problem (Asrizal et al., 2019). From the problems found, a solution is needed to solve them. Previous researchers have shown that STEM integrated learning is the best approach to overcome this problem and improve student learning outcomes.
Utilizing cutting-edge teaching strategies, such as STEM, is one way that educators may help kids develop their critical and creative thinking abilities Students can be guided and trained to think critically, creatively, and rationally using the stem approach. Apart from that, it also provides skills for students to be more evaluative in solving a problem (Lancaster et al., 2021). Integrating several components in the STEM approach such as teaching materials can make students carry out critical and creative thinking processes. Supported by research findings which show that the STEM approach can actualize increasing students' mathematical critical thinking skills (Makhmudah et al., 2021).
The results of previous studies have limitations. These limitations include, 1) the results obtained do not explain the effect of several similar studies of STEMintegrated Mathematics and Natural Science (MNS) teaching materials on students' critical and creative thinking skills, 2) only applying STEM-integrated MNS teaching materials at one educational level, 3) only using one STEM integrated subjects. Based on these limitations, this study wanted to integrate all existing similar studies to determine the effect of several similar studies of STEM-integrated MNS teaching materials on students' critical and creative thinking skills using the meta-analysis method.
Meta-analysis uses statistical techniques to combine the results of similar studies from two or more studies to obtain quantitative data sets (Anwar, 2005). Metaanalysis is done to research other people's studies and get the right data. In statistical analysis, results can be combined in meaningful ways. The results of each study were converted into an effect size to allow comparisons between them.
Meta-analysis research was chosen as a research method for several reasons. First, many articles discussed the effect of STEM-integrated MNS teaching charts on students' critical and creative thinking skills. Second, meta-analysis research did not depend on school conditions thereby reducing the risk of research delays. Third, the impact of STEM-integrated MNS teaching materials on students' critical and creative thinking abilities has not yet been studied. Fourth, it is not yet known the effect of a number of similar studies using STEM-integrated MNS teaching materials on students' critical and creative thinking skills which have a significant effect based on educational level, subjects and media used.
The critical and creative thinking abilities of students can be raised by incorporating STEM into MNS teaching materials. Numerous earlier studies have established it in this way, however they came to various conclusions. In light of this, it is crucial to carry out metaanalysis study on how STEM-integrated MNS teaching materials affect students' abilities to think critically and creatively. The goal of this study is to determine the degree to which students' critical and creative thinking abilities are impacted by the STEM integrated MNS teaching materials in terms of educational level, subjects, and media employed.

Method
This research uses meta-analysis method. Research that is undertaken through the summarization, review, and analysis of data from numerous previous studies is known as meta-analysis (Pangesti et al., 2017). The information gathered is secondary information, which was gleaned from articles published in national and international journals that summarized the findings of earlier studies. The following measures were done in this study: Create a study question first, focusing on how STEM-based instructional materials affect students' critical and creative thinking abilities. Second, determine 22 national and international journal articles published from 2019 to 2022. The criteria for journal articles analyzed were in the form of articles used to review the influence of STEMbased teaching materials, the moderator variable in this study had to be related to students' critical and creative thinking skills, and the journal articles used had been accredited as evidenced by the ISSN and DOI. Third, the researcher uses Cohen's equation to determine the effect size of each article and uses the random effect model and fixed effect model to determine the overall effect size.
Fourth, the effect size values obtained are averaged according to the moderator variable. Fifth, the effect size values obtained are categorized based on the effect size categories in Table 1 (Dincer, 2015).

The Effect of STEM-Based Math Teaching Materials on Critical and Creative Thinking
Based on data from 22 articles obtained from 2019 to 2022, an effect size calculation is carried out to determine the effect of STEM teaching materials on students' critical and creative thinking skills. The researcher calculated the effect size based on 4 criteria, namely based on the level of the educational unit, subject units, skills and teaching materials. STEM-based teaching materials can develop creative thinking skills in the aspects of fluent thinking, flexible thinking, detailed thinking and original thinking. Overall, 22 articles obtained the effect size of the influence of STEM integrated teaching materials on critical and creative thinking skills as shown in Table 2. Based on the heterogeneity test, it was found that the value of Q > df, then the estimation of the variance between articles is quite large and heterogeneous. The model that is suitable for calculating summary effect sizes is the random effects model. The heterogeneity value of the article data is 96.80%, indicating that there is a population difference between articles of 96.80%.
The summary effect size of STEM-based math teaching materials on critical and creative thinking abilities may be calculated using the random effect, as shown by the results of the heterogeneity testing that has been done. Table 3 displays the calculated summary effect sizes on students' critical and creative thinking abilities.
Based on the results of hypothesis calculations on students' critical and creative thinking, it was found that the 22 articles used showed that STEM-based teaching materials had a significant effect. The results of the weighted summary effect size obtained were 2.1 indicating that STEM-based teaching materials were in the very high category with confidence intervals below 1.08 and above 2.69. The results of hypothesis testing also show that the p value <α, which shows that testing the hypothesis Ho is rejected. The results of Ho being rejected show that as many as 22 similar articles have the effect of STEM-based teaching materials on critical and creative thinking skills.

The Effect of STEM-Based MNS Teaching Materials on Critical and Creative Thinking Skills Based on the Level of the Educational Unit
Based on the results obtained in table 4 it is known that different levels of educational units can be tested for heterogeneity. All MNS teaching materials used by each educational unit were tested for heterogeneity, because the population of the educational units was different. The two educational units that were tested based on the articles obtained were the junior and senior high school education units. Based on the results of heterogeneity testing of two educational units that used STEM-based MNS teaching materials, the value of Q > df was obtained, so the estimation of variance between articles was quite large and the data was heterogeneous. The model that is suitable for calculating the summary effect size for the three-learning media is the random effects model. The heterogeneity values in the two educational units from the data are 96.39% and 97.05% so that it can be seen that there are population differences between articles in each educational unit. The next stage is testing the hypothesis based on educational units.
Based on the hypothesis testing contained in table 5, it is known that the results of the effect sizes of the two educational units are in the very high category. Testing the hypothesis on 2 educational units yields a p value <α, with a value of p = 0.00 then testing the hypothesis H0 is rejected. The test results show that there is an influence of STEM-based MNS teaching materials on critical and creative thinking skills in junior high and high school education units. Previous research by Karunia (2022) states that high school students' critical thinking skills can be improved using STEM integrated teaching materials. Another study by Binar (2020) states that using STEM-based worksheets can improve students' critical and creative thinking skills.

The Effect of STEM-Based MNS Teaching Materials on Critical and Creative Thinking Skills Based on Subject Level
Based on the results of heterogeneity testing of 4 subjects using STEM-based MNS teaching materials, the value of Q > df is obtained, so the estimation of variance between articles is quite large and the data is heterogeneous. The model that is suitable for calculating the summary effect size for the three-learning media is the random effects model. The heterogeneity values in the four subjects from the data were 96.85%, 96.26%, 97.54% and 98.27%, so it can be seen that there are population differences between articles in each subject. The next stage is testing the hypothesis based on the subjects.
2 Based on the hypothesis testing contained in table 6, it is known that the results of the effect sizes of the four subjects are in the very high category. Testing the hypothesis on 4 subjects resulted in a p value <α, with a value of p=0.00 for Mathematics, Physics, and Science subjects and p=0.01 for Biology subjects. Then testing the hypothesis Ho is rejected. The test results show that there is an influence of STEM-based MNS teaching materials on critical and creative thinking skills in junior high and high school education units. Relevant research by Ramli et al. (2020) states that STEM-based physics worksheet teaching materials can improve students' critical thinking skills during the learning process. Another researcher Rasyid et al. (2021) stated that the application of STEM-based worksheets can improve students' creative thinking in biology subjects about the concept of the human nervous system.

The Effect of STEM-Based MNS Teaching Materials on Critical and Creative Thinking Skills Based on Skills
The third result is in the study of the effect of STEMbased MNS teaching materials on critical and creative thinking skills based on skills. There are 18 articles discussing the influence of STEM-based MNS teaching materials on critical and creative thinking skills based on critical thinking skills and 5 articles on creative thinking skills. Summary effect sizes were obtained through heterogeneity testing beforehand. Testing heterogeneity on critical and creative thinking skills can be seen in Table 8. The critical and creative thinking abilities reveal Q > df, which means that the estimation of the variation between articles is quite large and the data is heterogeneous, according to the results of the heterogeneity test. The model that is suitable to be used to calculate the size of the summary effect on the two skills is the random effects model. The value of heterogeneity in critical and creative thinking skills is 97.15% and 93.05%, so that it can be seen that there are population differences between articles in each skill.
Critical thinking is an indirect need of teachers and students while using the STEM approach. One of the traits of critical thinking is the ability to discover one's own ideas. The Student Worksheet is one of the resources available to enhance instruction that can help students' critical thinking abilities (Santoso et al., 2019). According to Novitasari et al. (2022) Student Worksheet functions to assist students in the learning process and assist students in practicing critical, creative and innovative thinking skills. STEM integration involves teaching science, technology, engineering, and math together to foster students' creativity as they work to solve issues in real-world situations (Sukmagati et al., 2020). The hypothesis regarding the impact of STEMbased MNS teaching materials on critical and creative thinking skills based on skills is being tested in the following stage. Table 9 displays hypothesis calculations based on abilities. Based on the results of hypothesis testing in table 9, it is known that the results of the effect size on critical and creative thinking skills belong to the very high category, namely 1.96 and 2.65. Testing the hypothesis on critical and creative thinking skills has a p value <α, with a value of p = 0.000, the Ho hypothesis testing is rejected. So that the test results show that there is a positive and significant influence between STEM-based MNS teaching materials on critical and creative thinking skills. Relevant research by Dewi (2019) states that STEM-based Student Work Sheets on work and energy materials are more suitable for use to identify and improve critical thinking skills of high school students. Another study by Purwaningsih et al. (2021) states that STEM-based student worksheet is effective for increasing students' creative thinking abilities.
Based on the type of teaching materials, the impact of STEM-based MNS teaching materials on students' critical and creative thinking abilities. The study of the impact of STEM-based MNS teaching materials on critical and creative thinking abilities based on the type of teaching materials produced the fourth outcome. There are 14 teaching material articles on worksheets, 3 module teaching material articles, and 5 other teaching material articles that discuss the impact of STEM-based MNS teaching materials on critical and creative thinking abilities dependent on the kind of teaching materials.
Summary effect sizes were obtained through heterogeneity testing beforehand. Table 8 displays the results of the heterogeneity testing based on the type of instructional materials.
The types of teaching materials exhibit Q > df, according to the heterogeneity test results, so the estimation of the variance between articles is quite large and the data is heterogeneous. The model that is suitable for calculating the size of the summary effect on the three types of teaching materials is the random effect model. Heterogeneous values on student worksheets and modules are 96.21% and 97.44%, so it can be seen that there are population differences between articles in each teaching material.
An essential component of putting education into practice is the use of teaching materials. The teacher will have an easier time carrying out instruction thanks to the teaching resources, and pupils will be more cooperative and find learning to be simpler. Teaching materials are a collection of teaching materials arranged systematically which represent concepts that direct students to achieve a competency (Magdalena et al., 2020). Integration of STEM components can improve students' academic performance, particularly in terms of improving learning achievement in science and technology (Pangesti et al., 2017). The following step is to conduct an experiment to determine whether STEM-based MNS teaching materials have an impact on students' ability to think critically and creatively. Table 11 shows the calculation of the hypothesis based on the kind of instructional materials. The findings of the effect size on the type of teaching materials are known to be categorized as very high categories based on the results of the hypothesis testing found in table 11, namely 2,095 and 2,113. Testing the hypothesis on the type of Student Worksheet and module teaching materials shows that the value of p < α, with a value of p = 0.000, the H0 hypothesis test is rejected. So, the test results show that there is a positive and significant influence between STEM-based MNS teaching materials on critical and creative thinking skills based on the type of teaching material. According to pertinent research by Hasanah et al. (2021), combining the PBL learning model with STEM-based LKPD can help high school students' critical thinking abilities. According to a different study Rusyati et al. (2020), using STEM-based e-modules can help students' critical thinking abilities.

Conclusion
Based on the data and results of the meta-analysis research, several conclusions can be stated. First, STEMbased MNS teaching materials with a variety of educational units stated that there was a significant influence on critical and creative thinking skills. Second, STEM-based MNS teaching materials with a variety of subjects have a significant influence on critical and creative thinking skills. Third, STEM-based MNS teaching materials have an influence on students' critical and creative thinking skills. Fourth, STEM-based MNS teaching materials with a variety of teaching materials have a significant influence on students' critical and creative thinking skills. This research can be a reference in developing STEM-based teaching materials as research relevant to students' critical and creative thinking skills.