Development and Validation of a Test of Science Process Skills for Secondary Students on Cellular Living System Organization Matter: Rasch Model Analysis

: This study aims to develop and implement the instrument of science process skills tests on cellular living system organization matter, explicitly designed for secondary students. The indicators developed include formulating hypotheses, communicating, applying concepts, asking questions, observing, planning experiments, using laboratory tools, classifying, and interpreting data. The steps of this research are theoretical construction, determining the purpose of the assessment, constructing indicator items, compiling the items, expert judgment, instrument revision, field trials, and finalization of the instrument. The Science Process skills test consisted of 27 multiple-choice questions. The subject was 30 secondary school students. Data were analyzed using Rasch analysis. Evidence of the validity and reliability of the tests are presented. The conclusion is that the science process skills test instrument is theoretically and empirically valid and reliable.


Introduction
The rapid growth of science and technology in the 21 st century has significantly changed the global landscape (Marburger, 2011), including education (Ramaila & Molwele, 2022).The current generation is very dependent on technology (Lemley et al., 2014) because the youth generation lives surrounded by technology, and they learn much with the technology around them.Technology, including independent learning, has made the current generation independent (Schwarz et al., 2014).Meaningful education emphasizes student independence in acquiring knowledge according to their developmental age.The current internet era reduces student dependence on learning and shifts the teacher's role (Gentile et al., 2023;Szymkowiak et al., 2021).Independent learning can be well implemented if the supporting factors can be fulfilled, such as more precise guidance & tasks and in-course collaborative support among students (Hockings et al., 2018).
Today's science education is created to help students comprehend scientific ideas and master 21stcentury skills (Stehle & Peters-Burton, 2019;Turiman et al., 2012;Wan Husin et al., 2016).In Indonesia, the science curriculum has undergone a few waves of reformation, from the Leer plan in 1947 as the first curriculum to the Merdeka curriculum in 2022 as the latest.The Merdeka curriculum strongly emphasizes the accepted practices of the scientific learning methodology and shapes the behavior of active and inquiry-based learning.This approach requires education designed through a scientific method based on laboratory activity.In this respect, scientific process skills (SPS) are essential in teaching ways of reaching knowledge and have become an important aim in science education.On this account, SPS learning has become integral to science curricula at all levels in many countries.It has also become one of the recent approaches regarding giving science education more efficiently to students.
Science process skills are complex abilities scientists use to conduct scientific investigations (Manu & Nomleni, 2018).Science process skills, hereafter referred to as SPS, are essential skills in thinking and conducting studies (Hodosyová et al., 2015) which are necessary for students to master to be ready to compete in the era of globalization (Budiyono & Madura, 2016).Science process skills reflect the methods used by scientists in producing complete information about science, such as products, attitudes, processes, and applications.SPS is an active action such as making observations, identifying problems, and predicting what students can develop through exercises in science learning.In science learning students are challenged to balance between science concepts and process skills (Hutahaean et al., 2017).SPS is essential to be trained in education because it can help students find ideas based on scientific stages that they do themselves so that students become more remembered and satisfied with the ideas they find (Winandika, 2020).
In practicing SPS, it must collaborate with specialized knowledge by presenting problems that must be solved through scientific method activities for these skills to be applied.It is not valid to assess process skills in tasks that require conceptual understanding that is not available to students.For this reason, it's crucial to evaluate only process abilities for topics where conceptual knowledge won't be a barrier to employing them.In all cases, a skills assessment is influenced by the ability to use the skills and the knowledge and familiarity with the subject matter on which the skills are used.
There are several developments to assess SPS.Among them are tests developed by (Burns et al., 1985;Dillashaw, Gerald F., Okey, 1980;Ludeman, 1975;Molitor & George, 1976;Tannenbaum, 1971).Meanwhile, some indicators of students' science process skills scores in Indonesia are still relatively low (Hartono et al., 2022).Teachers usually use science process skills assessments that are not developed from the Indonesian science curriculum.However, there has not been much research on the development of science assessment of students' process skills based on the science curriculum in Indonesia, especially for high school students.Therefore, this research aims to develop science process skill questions that are integrated with the Indonesian science curriculum so that measurements become more objective.The research reported here attempts to build a multiple-choice test of the integrated processes of science.Reliability and time efficiency are the primary reason for such a test.However, a primary consideration must be validity.The test score must accurately assess the student's ability to successfully perform the process in question.

Method
The SPS test was developed by verifying the previous works on developing the SPS test for science (Jalil et al., 2018;Ong et al., 2016).The instrument development was adapting (Gerald Dillashaw & Okey, 1980).
The SPS test instrument was created with 27 multiple-choice questions as the initial product.Three experts validated the Content SPS test instrument to evaluate each test time representing nine SPS indicators.The validation sheets were analyzed using descriptive analysis calculated in percentage representation (Harlen, 1999).
After revising several times according to expert suggestions, the instrument was tested on 30 secondary school students who were obtained using purposive sampling.Data were analyzed using Rasch analysis with Ministep software 5.6.0.0 version to determine the validity, reliability, and item analysis of science process skills tests.The design of this study is displayed in Figure 1.

Identifying the Test Objective and Specifying the Content
The purpose of this test is to develop a quality instrument in terms of validity, reliability, and items analysis to assess nine indicators of basic science process skills of science process skills.This instrument should be considered suitable for junior high school students.The summarizes of the nine science process skills with their respective descriptions presenting in table 1. Recognizing that there is more than one possible explanation for an event Realizes that one description needs to be tested by obtaining more evidence or solving problems Using Laboratory Tools

Writing Test Items
An important consideration in writing science process skills questions is the test format.The type of test used is multiple choice with four answer options.Review of multiple choice test selection because it is relatively short, easy to assess, objective, and can reduce assessment errors.The items in the developed test are Content free.Table 2 summarizes the items that are appropriate for each Science Process Skill.The following example of question development can be seen in Figure 2.

Determine Test Items Analysis Item-Person Map Analysis
An item-person map is a variable map that displays the distribution of test takers' abilities and the item difficulty level on the same scale.This variable map provides information on how the items that make up the test developed are feasible to measure students' abilities.From the variable map, three types of item groups are obtained, namely the group of items that cannot be reached by students with the highest ability with a logit value >2, the group of items that can be matched by all abilities of students located at a logit value of -2 to 2, and the group of items that have a difficulty level less than the lowest ability of students with a logit value <-2.The question is declared feasible if the question can reach all students' abilities and students can reach all levels of difficulty of the question items.From Figure 3, it can be seen that students' abilities are pretty evenly distributed.Students with codes 04, 07, 15, 16, and 29 have high ability levels, while students with codes 08 and 10 have low ability, and the rest are students with moderate ability categories.Figure 3 also explains that item number 12 is the most challenging item with a logit value >2.Students with high ability categories also have a probability of not being able to answer the question correctly.Furthermore, 24 items have a logit value of -2 to 2. These items have a difficulty level that varies from easy to difficult and is evenly distributed on the same scale as the entire ability of students.Questions in these categories are questions with numbers 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26 and 27.Meanwhile, questions 13 and 17 are too easy with a logit value <-2 because all students can have a probability of answering correctly, including students with low ability.

a. Analysis of the Conformance Level of Items
Analysis of the conformance of question items using Rasch analysis can be viewed from the standardized z value (ZSTD) and the outfit Mean Square (MNSQ) value.The standard criteria for item analysis, according to (Boone et al., 2014) is that the question is feasible if the MNSQ value is in the range of 0.5 -1.5, and the ZSTD value is in the range -2.0 -2.0.The following are the results of Rasch's analysis related to the suitability of items on the developed science process skills test (Figure 4).Based on the analysis above, it is known that the MNSQ value is 0.90, and the ZSTD value is 0.00.Based on the criteria (Boone et al., 2014), the items developed were appropriate.Meanwhile, (Bond & Fox, 2015) explained that the ideal MNSQ value expected through Rasch analysis has a value of 1. Values less or more than 1 indicate variation in test performance.Based on the data above, the MNSQ infit value is 1.01, which indicates that the data has 0.01% more variation, and the MNSQ outfit value is 0.90, which indicates that the data has 10% less variation.Meanwhile, the ZSTD value is expected to be close to 0 (zero).Based on the data above, the ZSTD infit value is 0.06, and the ZSTD outfit value is 0.00, which means the ZSTD value is still close to 0 (zero).The analysis shows that the questions are appropriate.

Items, Dimensionality, Validity and Diffuculity
Dimensionality is important to evaluate whether the multiple choice test instrument developed can measure what should be measured.From the results of this analysis, the Raw variance measurement result is 36.24%.This shows that the minimum undimensionality requirement of 20% can be met.Thus, the test instrument developed in this study is valid enough to measure students' ability.The following analysis is an item fit analysis to measure the validity of each item.One of the main item fit statistics is the infit mean square (INFIT MNSQ).Infit mean square measures the consistency of learner fit with the item characteristic curve for each item with consideration given to a close person with a probability level of 0.5 (Alagumalai et al., 2005).Based on the table of difficulty levels in the figure 5, all items have infit MNSQ and outfit MNSQ values with an acceptable range.
Figure 5 also shows the order of difficulty of the questions.The more numerous the measure, the more complex the questions.Based on (Sumintono, 2015) categorizes the difficulty level of the item into four categories based on the measured value, namely very easy (less than -1), easy (-1 to 0), difficult (0 to 1), and very difficult (more than 1).Based on Rasch analysis data per item in the table above, there are two questions in the very difficult category (numbers 12,11,24,7 and 23), Eight questions in the difficult category (numbers 4, 25, 16, 18, 22, 1, 2, and 8), six questions in the easy category (numbers 3, 6, 10, 14, 21 and 20), as well as six questions in the very easy category (numbers 9, 19, 15, 5, 13 and 17).

Reliability Analysis
The following analysis is the measurement of separation and reliability.This analysis analyzes the distribution of items and persons presented in Figure 3. Reliability analysis Based on (Cordier et al., 2018) an instrument is said to be good when it has a reliability of more than 0.8.Based on these data, the science process skills test item has a reliability value of 0.87.This value indicates that the item questions have varying difficulty ranges.However, for the person (research respondents), a reliability value of 0.79 indicated that the respondents were homogeneous in their ability level.

Conclusion
The results of the analysis of the ability of test participants show that students have varying abilities.This variation is beneficial for research because it can be seen how the ability of each category of student ability to answer questions.The results of the item analysis show that all question items are categorized as empirically valid because they meet the standard criteria set.The developed question items are also in accordance with students' abilities, meaning that difficult questions will only be answered correctly by students with high abilities, while students with low abilities will tend to answer incorrectly.The results of the analysis also show variations in the level of difficulty of the questions ranging from very easy, easy, difficult and very difficult questions.This indicates that the test instrument developed has good validity.

Figure 1 .
Figure 1.The design of the study Using tools and/or materials Knowing the reasons why using tools or materials Planning Experiment Determine the tools, materials, or sources to be used Define the variables or determining factors Decide what will be organized, observed, recorded Determine what will be done in the form of work steps Applying Concept Using concepts or principles that have been learned in new situations Using concepts in new experiences to explain what is happening Interpreting Making connections between observations Finding patterns or regularities in a series of observation concluding Communicating Describing empirical data from experiments or observations with graphs or tables or diagrams Compiling and submitting reports systematically and clearly Explaining the results of an experiment or research Reading graphs or tables or diagrams Discussing the results of activities on a problem or an event Asking Question Asking what, how, and why Asking for explanations Asking questions with a hypothetical background

Figure 4 .
Figure 4. Analysis of ZSTD and MNSQ tests of science process skills for student

Table 1 .
Summarizes the Nine Science Process Skills with Their Respective Descriptions

Table 2 .
Number of Items for Each Component of Science Process Skills