STEM-PjBL Learning: The Impacts on Students’ Critical Thinking, Creative Thinking, Communication, and Collaboration Skills

: The purpose of this study was to analyze the effectiveness of STEM-PjBL learning on students' critical thinking, creative thinking, communication


Introduction
Today's development has been entering the new era of Industry 4.0, marked by using technology as the primary basis in life. One of the essential elements that must be considered in the industrial revolution 4.0 is to prepare graduates with 21st-century skills. Partnership for 21st Century Skills (2017) mentions that the required skills include critical thinking and problem-solving skills, creativity and innovation, collaboration and communication, or the 4C. Critical thinking is a skill that needs to be learned by students as it is closely related to their deep understanding of content or problems in real life (Tiruneh et al., 2018). Pedrosa-de-Jesus et al. (2014) asserted that this skill is one of the highest cognitive abilities and is recognized as the main competency in education, especially for science and technology. This skill positively impacts students' problem-solving abilities and self-confidence (Azizi et al., 2018).
In addition, another skill that needs to be acquired by students is creative thinking skill. According to Birgili (2015), Creative thinking is a series of cognitive activities in students responding to an object, problem, condition, or the like, which aims to solve a problem. Collaboration with communication skills is another competency that should be mastered and honed by students (Kivunja, 2014). Good communication will allow students to be active in the learning process and ease them to understand information. Collaboration can build practical communication skills by placing them in the interpersonal part of students, making 4C skills related to one another.
According to preliminary research done at five schools in the form of filling out a questionnaire about the learning process they had received, 47% claimed that the instructor utilized the lecture technique, 37% the discussion method, and 22% other ways. Some teachers' lesson plans focused on the 4C features, but the learning process did not clearly identify the steps for improving students' 4C abilities. As a result, the teacher's learning process has not increased students' 4C abilities. According to student questionnaires on projects, 76% of students said they had never planned a project in 5008 groups, while 24% said they had designed a project but had yet to do it individually. According to the statistics, project-based learning should improve students' thinking abilities and creativity in project design. Santos (2017) states that the main problem in developing students' critical thinking in science is that science learning focuses on content and memorization abilities rather than creativity, meaningful understanding, and critical thinking. 4C skills are very important for students to meet the Industrial Revolution 4.0, so the learning process must train students' 4C skills.
Critical, creative, communicative, and collaborative thinking skills can be trained by learning using the science, technology, engineering, and mathematics (STEM) approach. STEM integrates the four disciplines of science, technology, engineering, and mathematics in an interdisciplinary approach and is applied based on real-world contexts. STEM education prepares students to hone higher-order thinking skills and attracts students' interest in learning, which is crucial in adapting to the competitive era (Wahono et al., 2020). The characteristics of STEM learning are very relevant and compatible with the 2013 curriculum, in which one of the improvements to the mindset developed by this curriculum is strengthening multidisciplinary and critical learning patterns. Science learning with a STEM approach trains students in critical, creative, collaborative, and communicative thinking. Therefore, learning with the STEM approach supports achieving skills in the 21st century. STEM-PjBL can be a solution to train students' thinking abilities and ways of thinking critically, creatively, collaboratively, and communicatively.
STEM learning has been developed in many countries for various reasons, including increasing scientific developments, global economic developments, individual educational needs, and the need for innovation in various fields of life (Ritz & Fan, 2014;Yildirim, 2016). Meita et al. (2018) show that applying STEM-PjBL can improve student learning outcomes in psychomotor aspects and creativity. Applying STEM can develop student creativity and curiosity (Lou et al., 2017). Soros et al. (2018) also show that applying STEM learning encourages students to think critically, have problem-solving skills, and improve student achievement. Most previous research has not comprehensively examined the application of learning with a STEM approach that is integrated with projects on students' 4C skills. Triana (2019) has examined the effectiveness of STEM-PjBL learning on 4C skills in class X of Senior High School on environmental change material. The characteristics of students at each level are different. Thus, in this research, we want to examine the application of STEM-PjBL learning to train high school students in critical, creative, communicative, and collaborative thinking skills in class XI on the Respiratory System material. Based on this background, this study examines the effectiveness of STEM-PjBL learning on students' critical, creative, communicative, and collaborative thinking skills (4C).

Method
This research is quantitative. The samples in this study were 106 students from class XI of State Senior High School 16 Semarang in the 2019/2020 academic year. The purposive sampling technique took them. Data were collected using critical thinking skills questions and observation sheets. This research design used a one-group pretest and posttest design (Cresswell, 2013), as presented in Table 1. The device developed for STEM-PjBL learning of the respiratory system meets the valid and practical criteria for use in learning (Kurniahtunnisa et al., 2020). Before the learning process, preparation was done by compiling some learning tools: a syllabus, lesson plans, student worksheets, and teaching materials. The learning tools used were from development that had been valid and practical (Kurniahtunnisa et al., 2020). Students were given pretest questions before the learning process and posttest afterward. The test was in the form of multiple-choice questions made by researchers validated by evaluation experts. There were 25 multiple-choice questions from C4 to C6 developed based on critical thinking indicators, according to Ennis (2016).
In addition to critical thinking skills, data on evaluation scores from the pretest and posttest were used to determine the increase in student learning outcomes, which were analyzed by the N-gain test and paired sample t-test using SPSS 28. The following formula can calculate the increase. Description: The total of the g factor is then compared with the Gain criteria as follows in Table 2.
Qualitative data from observation sheets on students' communicative, collaborative, and creative thinking skills were obtained from observer assessments during the learning process. The rubric for assessing students' creative thinking skills was developed based on indicators of creative thinking abilities from the Torrance Creativity Framework: fluency, flexibility, elaboration, and originality. Meanwhile, the rubric for assessing communication and collaboration skills was adapted from BIE (2015). The data were analyzed descriptively by percentage using the following formula 2.
Percentage (%) = total score obtained maximum score x 100% (2) The percentage obtained was then interpreted as sentences, as shown in Table 3.

Result and Discussion
The effectiveness of STEM-PjBL learning was determined based on test results and observations of students' critical, creative, communicative, and collaborative thinking skills.

Critical Thinking Skill
The effectiveness of learning was measured by the average score of increasing students' critical thinking skills before and after treatment using the STEM-PjBL learning device for respiratory system materials through the N-gain test, presented in Table 4. Then, the hypothesis was tested using the pairedsample t-test in the SPSS 28 program. The results of calculating the paired-sample t-test using SPSS 28 obtained a significance value/Sig. (2-tailed) of 0.001, indicating a difference in critical thinking skill scores before and after STEM-PjBL learning. Based on these findings, it was concluded that STEM-PjBL learning effectively trained students' critical thinking skills. This is because the STEM-PjBL learning process made students more active and encouraged them to think critically during the learning process through project design activities in groups. Students' critical thinking skills were also trained by giving descriptive questions with a cognitive level, namely Higher Order Thinking (C4-C6), on each student's worksheet during the learning process. Students discussed and answered these questions in groups to train their critical thinking skills. In addition, students were also facilitated to surf the internet to get more information in the group discussion process and eventually trained students' skepticism in sorting out the information. This method is supported by the research of Sidiq et al. (2021) that learning by habituation of HOTS-based science questions effectively increases students' critical thinking skills.
Overall, the improvement of students' critical thinking skills with the N-gain test was 0.67 with moderate criteria. This is because the learning process was only carried out in five meetings. One factor that influences students' critical thinking skills is the learning process at school. Critical thinking skills must be trained and cannot be obtained instantly. This aligns with Redhana's (2013), statement that critical thinking skills require continuous learning and practice. Soyomukti, (2010) also supported this by stating that critical thinking is not a skill that can develop independently. It must be learned and trained in formal education and everyday life. Santos (2017) added that the main problem in developing students' critical thinking in science is that the learning is carried out with a focus on content and memorization skills, not on creativity, meaningful understanding, or critical thinking.
Critical thinking skill data was also analyzed based on the critical thinking skill indicators proposed by Ennis (2016). The data are presented in Table 5. Based on the analysis of each aspect of students' critical thinking skills in Table 5, the aspects of providing simple explanations, concluding, and providing further explanations were on critical criteria. Meanwhile, building basic skills and providing alternative problemsolving were highly critical criteria. This is because, in STEM-PjBL learning, students were directed to provide alternative solutions to solve problems in everyday life. Students were directed to design solutions to problems related to the respiratory system, including air pollution, the dangers of smoking, and designing tools to improve lung performance. Before designing a project, students investigated air pollution levels, studied the dangers of smoking, and studied the vital capacity of the lungs so that they could directly identify problems in everyday life. From the results of this identification, students made projects in groups to overcome these problems to feel more enthusiastic about designing the project. Moreover, this learning also builds students' basic skills by looking for various literature sources to find solutions to existing problems. Further, students are directed to consider whether the sources obtained are reliable or not by comparing information from various works of literature, which simultaneously train students' critical thinking skills. This aligns with Siew et al. (2015); Sumardiana et al. (2019) that learning with the STEM-PjBL model makes students creative and develops their critical thinking skills in solving problems. Activities and tasks that involve real-life and problem-solving should be introduced and supported in their implementation (Amoako-agyeman, 2016). The students indicated they preferred to study science knowledge related to practical experience and agreed that science could be applied in everyday life through STEM-PjBL learning (Tsai et al., 2016).

Students' Creative Thinking Skills
The effectiveness of STEM-PjBL learning was measured based on students' creative thinking skills. The data from the observation of students' creative thinking skills are presented in Figure 1. Based on Figure 1, it can be concluded that STEM-PjBL learning effectively trains students' creative thinking skills with a high average criterion. This is because applied learning is contextual by integrating aspects of science, technology, engineering, and mathematics with various group project activities to make students more active in the learning process and meaningful. In STEM-PjBL learning, students create product designs and manufacture products that stimulate their creativity in creating products. This is in line with Iskandar et al. (2022); Siew et al. (2015); Yulaikah and Rahayu (2022) stated that learning with the STEM-PjBL model makes students creative and develops their critical thinking skills in solving the problems they face. Kennedy & Odell (2014) stated that a curriculum that engages students in STEM enhances learning strategies that challenge them to innovate and discover. Creativity helps students solve challenges, think exclusively, and act socially (Schoeneman et al., 2016). Project-based learning effectively improves students' science process skills and creativity (Fajrina et al., 2018;Milla et al., 2019;W. P. Sari, 2018).
Based on analyzing each aspect of students' creative thinking skills, creativity for fluency was in the high category. Most students could write down the project objectives because their projects related to problemsolving in everyday life. Holbrook & Rannikmae (2009) stated that science would be more accessible when what is learned is related to human life. However, students are unfamiliar with project-based learning that produces a product; therefore, this study applied didactic anticipation by providing an overview of the project investigation of environmental problems and the project solution to each group's problems. Brousseau (2002) supported this theory by stating that a teacher's actions in the learning process will create a situation that can be the starting point for the learning process. Factors that play a role in implementing PjBL learning are student support, teacher support, practical workgroups, and balancing didactic instruction with independent inquiry work methods (Kokotsaki et al., 2016).
Creativity in terms of flexibility and originality was in the moderate category. The originality aspect was assessed based on the student's ability to create a work or project. The projects created contained elements of novelty and uniqueness. Based on the study's results, the projects made by students showed a bit of those two characteristics they created. This can be seen in the originality aspect's indicators that have not been met. It happened because students had a tendency to depend on the teacher and the guidelines given during the learning process, which can hinder the flexibility of thinking in developing creativity. The limitations of students in self-development and creativity were due to Fluency Flexibility Originality Elaboration Synthesis Indicator of CreativeThinking Skills dependence on the guidelines provided (Argarini et al., 2014). Creativity in the aspect of detail (elaboration) was in the moderate category. Students could describe tools and materials and provide information on each part of the project because they were familiar with them, as the tools and materials are usually used in their daily lives. However, during the project presentation, only a few students in one group could explain the work steps correctly. This was because students occasionally relied on their group friends to do the work; hence they did not understand what they were doing. This aligns with Sari et al. (2013), who emphasized this student's habit in their research, stating that the low detailing skills were caused by students' habit of putting the weight of group projects on their groupmates during practice. Hence only students serious about practicing would understand the details about it.
Creativity in integration (synthesis) was in the high category. Students could create products that function according to their goals and explain their working procedures. This is because students understood the purpose of making and planning group projects. Before designing a project, each group needed input and approval from the teacher so that students understood better and took responsibility for the project. Applying STEM in learning can encourage students to develop and utilize technology, sharpen their cognition, and apply knowledge (Khaeroningtyas et al., 2016). Applying STEM in learning can encourage students to be active in the learning process and increase student motivation in learning related to the STEM field. This is supported by Izzah & Wiyanto (2018), which show the effect of STEM learning on the positive attitudes of high school students in the STEM field.

Students' Communication Skills
The effectiveness of STEM-PjBL learning was also measured based on students' communication skills. Observational data are presented in Figure 2.

Figure 2. Assessment Results of Students' Communication Skills
The average student's communication skills were on the high criteria. This was because, in the learning process with STEM-PjBL, students communicated with each other in designing and creating projects and making project presentations in groups. The ability to express ideas by presenting them in front of the class made students accustomed to oral communication. The results of research support this by Wijayanti & Fajriyah (2018) that the implementation of STEM-PjBL can improve students' scientific work skills, especially in communication and interpretation, as well as planning experiments. This is in line with the research of Triana et al. (2020) that STEM-PjBL learning is effective in students' communication and collaboration skills on environmental change materials.
Project-based learning that involves intense communication and collaboration throughout the project is an excellent way to develop communication and collaboration skills (Capraro et al., 2013). Further, the aspect of explaining ideas and information and organization was in the very high category. It happened because students clearly understood the investigative project and the product made, so they presented the information or arguments clearly, concisely, and logically; could be easily understood by other students; could sort essential things out for presentation; delivered a clear and exciting presentation.
More, the aspects of eyes and body, response to questions, and group participation were in the high category. This was because students clearly understood the investigative project and the products made, so they were more confident during the presentation. Students spoke clearly, had more eye contact with other students, could answer questions clearly and thoroughly, asked for clarification, and admitted when there was something they did not know by saying "I don't know" or explaining how the answer might be found when they were unable to answer. PjBL learning based on everyday problems and integrated with the STEM approach is a collaborative teaching strategy that has been proven to improve student communication skills (Husin et al., 2016). However, not all students were actively involved in the presentation process. This was because not all group members contributed significantly during the learning process. The main problem in group work is that group members do not contribute significantly (Hall et al., 2012;Lipson et al., 2016).
Communication skills are essential in 21st-century learning. Good communication will allow students to be active in the learning process and ease them to understand information. The results of Blume et al. (2013)  to one another. Dedovets & Rodionov (2015) added that learning with a STEM vision makes students have their knowledge, feel confident in conveying information, develop communication skills, use critical and creative thinking, help establish learning goals and research skills and gain new knowledge.

Students Collaboration Skills
The effectiveness of STEM-PjBL learning was also measured based on students' collaborative thinking skills. The research data are presented in Figure 3.

Figure 3. Assessment Results of Students' Collaboration Skills
The average of students' collaboration skills was in high criteria. This was due to the use of STEM-PjBL in the learning process. Students designed, made group projects and delivered the project presentations. They designed their products with effective time management to finish the product within the given timeframe (Sumarni, 2015). Thus, it took good cooperation and collaboration to achieve common goals. This is supported by Singer et al. (2020), who stated that working in groups in an authentic learning environment helps students improve their communication and collaboration skills. Project-based learning with a STEM approach improves students' collaboration skills because each group has to organize their work, materials, and individual assignments and manage their time. Therefore, in STEM-PjBL, students take responsibility for their learning and develop collaboration skills (Capraro et al., 2013). Students in groups will be encouraged or required to work together on a task and must coordinate to complete it. They will depend on each other to get a reward.
Implementing performance appraisal in STEM-PjBL can investigate student creativity and collaboration (Rustaman et al., 2018). The value of students' collaboration skills was obtained through observation during the learning process. The average of students' collaboration skills was in high criteria. This was because, in the STEM-PjBL learning process, students designed and made projects in groups. Project-based learning with a STEM approach improves students' collaboration skills because each group has to organize their work, materials, and individual assignments and manage their time. This aligns with Woro (2015), who stated that students would design their products and set effective time management to complete the product on time. Collaboration skill is related to student academic achievement. Putri et al. (2018) research emphasized this by showing a positive correlation between collaboration ability and student test results.
Furthermore, carrying responsibility for oneself, helping groups, and working as a team was very high. It was because there was a good division of tasks that teamwork was more meaningful. Thus, in STEM-PjBL, students took responsibility for their learning and developed collaboration skills. In implementing the PjBL model, the optimal results were creativity, communication, and collaboration skills (Rochmawati et al., 2020). Participants in several studies expressed a positive view of teamwork. Research conducted by Crowder & Zauner (2013); Zhou (2012) describes how group members encourage collaboration and help each other when misunderstandings occur. Another study showed that 70% of students prefer group work because it encourages self-evaluation and initiates negotiations when disagreements arise (Papanikolaou & Boubouka, 2010). Adapting to teamwork settings, including favourable and conflict situations, is essential for future classroom and career situations (Ralph, 2015). STEM-PjBL can facilitate changes in the organization of living systems, creativity, and student collaboration (Rustaman et al., 2018).
Meanwhile, other aspects, such as respecting others, making and following approvals, and managing projects, were in the high category. It happened because there was a good division of tasks, so each student needed to contribute to the project, and the rules and agreements during group work were clearly defined. The achievement in the high category was based on the class average. However, not all students were happy with group work, and some students were less likely to contribute to the group. Several studies identified negative aspects of teamwork. The main problem was that team members did not contribute significantly (Hall et al., 2012;Lipson et al., 2007).

Conclusion
STEM-PjBL learning on the respiratory system materials effectively trains students' critical thinking skills with critical categories. The results showed increased students' critical thinking skills in pretestposttest assessment with moderate criteria based on the N-gain test. The paired sample t-test results in the SPSS 28 program obtained a significance value/Sig. (2-tailed) of 0.001, indicating a difference in critical thinking skills scores before and after STEM-PjBL learning. This learning also effectively trains students' creative, collaborative, and communicative thinking skills with high category.

Acknowledgments
The researcher extends sincere thanks to everyone who contributed to the success of this study. We think that our findings will be helpful in the future advancement of science learning.

Author Contributions
1&2 Author contributes to the theoretical and conceptual foundation for this research. 1&3 Author is involved in creating research and learning tools and designing this work until it is published. 2&3 Author is involved in validation, data analysis, and interpretation. 1 Author, play a role in writing original draft preparation, writing review and editing, and funding acquisition. 4 Author, participate in data interpretation, funding acquisition, and paper design till it is published.

Funding
The research is not receiving any funding support from any source.

Conflicts of Interest
The authors declare no conflict of interest.