Sentiment Analysis Naive Bayes Method on SatuSehat Application

: The SatuSehat application is an application that provides health services to users. This application is a development of the PeduliLindungi application which is used to handle vaccination history in the new normal era. Therefore, it is important to classify user reviews into positive and negative sentiments using the Naïve Bayes method. The use of this method because it can produce a model that is quite accurate and effective. The results of data collection in this study were 25,000 of which 18,359 were negative and 6,641 were positive. The results of the Naïve Bayes accuracy test are 97% with negative sentiment results, namely precision has a value of 98%, recall has a value of 98% and f1-score has a value of 98%, while positive sentiment results, namely precision has a value of 94%, recall has a value of 94 % and f1-score has a value of 94%. This study aims to classify user reviews of the SatuSehat application into positive and negative sentiments and assess the performance of the Naïve Bayes method regarding public opinion on the use of the SatuSehat application based on reviews from the Google Playstore application.


Introduction
Application is software made by a computer company to do certain tasks, for example Microsoft Word, Microsoft Excel. The application comes from the word application which means the application of application usage. (Saarikko et al., 2020) an application is an application, storing something, data, problems, work into a facility or media that can be used to apply or implement existing things or problems so that they turn into a new form without losing values. the basis of data matters, problems, and the work itself. so the application is a transformation of a problem or work in the form of things that are difficult to understand to be simpler, easier and understandable for users. So that with the application, a problem will be helped more quickly and precisely (Salvagno et al., 2023). Applications have many types, among them are desktop applications that operate offline and web applications that operate online. A web application is an application that uses browser technology to run applications and is accessed via a computer network, (Fauzan et al., 2022).
Meanwhile, Alsaffar et al. (2022) a web application is a program that is stored on a server and sent via the internet and accessed through a browser interface. From the above understanding, it can be concluded that a web application is an application that is accessed using a web browser via the internet or intranet network. A web application is also computer software coded in a programming language that supports web-based software such as HTML, JavaScript, CSS, Ruby, Python, PHP, Java and other programming languages. Around 1,200,000 mobile applications are currently available on the Google Playstore, thus offering users many choices. Therefore, it is important for the user to know the functionality of the application to be installed, what other consumers think about it, and then decide whether to install it or not. User ratings usually consist of two parts: the rating and the review.
In this case, the value of ratings and reviews can reach thousands or even millions which makes it 5525 difficult for application developers to find ways to improve application performance based on thousands of text comments (Alzubaidi et al., 2021). One of them is the SatuSehat application, which is an upgraded version of the PeduliLindungi application in the new normal era. The purpose of this application is to monitor people who have been vaccinated, need to carry out activities in crowded places (Nurmansyah et al., 2022). The SatuSehat application helps manage people's vaccination history and saves a history of places visited by people using the QR code scanning feature. In addition, the SatuSehat application provides registration opportunities for those who have not received vaccinations or do not know where to register for their vaccinations.
In its development, the SatuSehat application raises pros and cons which can be seen in the comments on the Google Playstore application. Given these problems, sentiment analysis is a perfect field study to process a collection of reviews and classify them into positive and negative reviews. Sentiment analysis, also known as opinion mining, involves processing data about an individual's opinion, behavior and emotions towards an entity (Shaik et al., 2023). These entities are individuals, or topics that are happening around us. The results of data processing can be in the form of information about how people view entities extracted from sentences (Yang et al., 2022).
The method that will be used to classify user reviews of the SatuSehat application is Naïve Bayes. Naive Bayes can provide better performance than other methods, if the number of samples available for model training is limited. This method tends to be more computationally efficient and requires less training data to produce a good model. In addition, the algorithm in Naïve Bayes is simple but has relatively high accuracy and speed (Diba & Nugraha, 2020). Several studies have been conducted previously using the Naïve Bayes method, such as research by (Salsabila et al., 2022) who conducted a sentiment analysis about the COVID-19 vaccine using the Naïve Bayes algorithm with a total of 3,780 data obtained. The results of the analysis show that the majority of user reviews are positive (60.3%), while neutral reviews (34.4%) outnumber negative reviews (5.4%). The accuracy value generated by the Naive Bayes Classifier algorithm is 0.93 (93%).
Other studies have also analyzed public opinion regarding vaccination with a total of 845 data by comparing the Naïve Bayes and SVM methods. The results obtained are that the classification of the Naïve Bayes method has an accuracy rate of 85.59%, which is higher than the SVM, which is 84.4% (Syahputra et al., 2022). Another research is research conducted by Illia et al. (2022), who conducted a sentiment analysis on the PeduliLindungi application with a total data of 4,636. The results of the Naïve Bayes classification show an accuracy value of 90% with a negative precision of 10%, a positive precision value of 99%, a negative recall value of 68% and a positive recall value of 91%.
The results of this study indicate that the positive reviews are higher than the negative reviews. However, this research was conducted in 2022 and before the application was developed into SatuSehat. Therefore, the purpose of this research is to classify user reviews of the SatuSehat application into positive and negative sentiments and assess the performance of the Naïve Bayes method regarding public opinion on the use of the SatuSehat application based on reviews from the Google Playstore application. In addition, it aims to display a visualization to find the words that appear most often in user reviews.

Method
Therefore, it is important to classify user reviews into positive and negative sentiments using the Naïve Bayes method This research flowchart is described as Figure 1.  Figure 1, show a research flowchart in which the first stage is data collection taken through user reviews of the SatuSehat application on the Google Playstore application. After that, text processing is carried out to select data so that it is more organized with the stages of case folding, text cleaning, tokenization, normalization, stopword removal and stemming (Syahputra et al., 2022). Case folding is the process of manipulating text by changing all uppercase letters to lowercase. Text Cleaning is a step to clean unnecessary text data, such as deleting null values, removing duplicate values, removing unwanted characters and symbols in text, and choosing which variables to use.
Tokenization divides a sentence or paragraph into individual word blocks, or what are called tokens. Normalization refers to the stage of changing words in abbreviated, colloquial and non-standard forms into standard forms. Stopword removal removes words which are usually many and are considered meaningless (Sarica & Luo, 2021) and stemming is the process of replacing words in basic words with affixes. The next stage is weighting using TF -IDF (Term Frequency Inverse Document Frequency). This process is carried out with the aim of converting text into numeric and counting how many words appear in the document. The results for TF -IDF are obtained from the results of TF multiplied and the formula can be seen in equations 1, 2 and 3 (Alfarizi et al., 2022).

Data Collecting
Features reviews from users is one of the features that Google Play has that can be used to see reviews from other users (Sutino & Siahaan, 2019). Apart from that, Google Play has a feature to analyze user reviews application making it easier for application makers to monitor the state of the application based on words or topics that appear frequently in related reviews design, privacy, profile, resource usage, speed, stability, updates, uninstall and usability of the application (Fuad & Al-Yahya, 2021).
The data used in this study came from reviews of the SatuSehat application on the Google Playstore application and for a period of three months, from January 2023 to March 2023. The data collected consisted of 25,00 reviews with 18,35 belonging to the negative class and 6,641 belonging to the positive class. The results of data collection are imported into the Jupyter Notebook software using python. In this process, rating data is converted into sentiment which can be seen in Table 1.
Changing the rating to sentiment in Table 1 aims to categorize text or data based on the sentiment contained in it, namely in this study, sentiment with a scale of one to three is defined as negative sentiment, and positive sentiment is on a scale of four to five. Sentiment analysis also uses an algorithm to process and classify built data. There are many algorithms can be used in analytical research sentiment. There are the usual top ten algorithms used, among them was C4.5 (Wankhade et al., 2022).

Pre-Processing
The stages in preprocessing are that the original data that has been collected will be further processed with the aim of cleaning the data, namely removing noise, clarifying features, and converting the original data according to research needs (Fan et al., 2021). In this study, pre-processing was carried out by case folding, text cleaning, tokenization, normalization, stopword removal and stemming. The first process is case folding which aims to make changes to letters that were previously mixed between capital letters and lowercase letters to become only lowercase letters. The case folding stage in this study can be seen in Table 2.
The second process is the text cleaning process which aims to clean sentences from link or hyperlinks, punctuation marks, mentions of username (@username), urls and numbers that should not be in the dataset. The stages of text cleaning in this study can be seen in Table  3.    ['error', 'sdh', 'uninstall', 'same', 'sj', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'onehealthy', 'doang', 'can', 'no', 'min', ' gausah', 'use', 'letter', 'gt'] The next process is the tokenization process which can be seen in Table 4. At this stage all the words that have been collected will have their punctuation removed such as symbols, characters and all that are not letters and the purpose is to separate sentences into words.
Tokenization is the separation or decoding of data textual into tokens, which are smaller means components (Verma et al., 2018). Principle of tokenization Konashevych (2020), namely separating each word that composes sentence in the document. It can be classified into sentence tokenization and word tokenization. The normalization process is a process that is carried out after everything has gone through the tokenize process. The normalization process is the process of changing words that were originally in the form of abbreviations or slang words back into standard words (Rianto et al., 2021). The normalization stage can be seen in Table 5.  ['error', 'sdh', 'uninstall', 'same', 'sj', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'satusehat', 'doang', 'can', 'no', 'min', ' gausah', 'use', 'letter', 'gt'] After Normalization ['error', 'already', 'uninstall', 'same', 'only', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'satusehat', 'doang', 'can', 'no', 'min', ' gausah', 'use', 'letter', 'that'] Database normalization is rarely performed on small databases, and is considered unnecessary for personal use (Albarak & Bahsoon, 2018). However, as the information contained in a database develops, the normalization process will be very helpful in saving the space used by each table in it (Shorten & Khoshgoftaar, 2019), as well as speeding up the process of requesting data. The purpose of database normalization is to reduce data repetition Arini et al. (2019), ensure data is in the right position, no amount of data enters the database, there is deletion of inappropriate data and data that is changed inappropriately. At the stopword removal stage, if there are conjunctions or irrelevant words, they will be removed, such as first, second and third person pronouns, a name, conjunctions and question words. These words are words that have no meaning when separated from other words and are not related to adjectives related to sentiment which can be seen in Table 6.  ['error', 'already', 'uninstall', 'same', 'only', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'satusehat', 'doang', 'can', 'no', 'min', ' gausah', 'use', 'letter', 'that'] After Stopword Removal ['error', 'already', 'uninstall', 'same', 'only', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'doang', 'can', 'use', 'letter', 'so'] The last process is stemming, which aims to replace word forms into basic words according to the structure of the Indonesia Dictionary. such as words that have initial and final affixes such as me, mem, meng, meng, di, per, ber, an, kan, i, nya etc. The affix words will be changed to basic words as shown in Table 7.  ['error', 'already', 'uninstall', 'same', 'only', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'through', 'application', 'doang', 'can', 'use', 'letter', 'so'] After Stemming ['error', 'already', 'uninstall', 'same', 'only', 'result'] ['if', 'show', 'certificate', 'vaccine', 'only', 'via', 'application', 'doang', 'can', 'use', 'letter', 'so'] Weighting The weighting will be carried out using Term Frequency -Inverse Document Frequency (TF-IDF). TF-IDF which aims to give weight to the relationship of a word (Term) to a document before doing calculations on the algorithm and overcome problems in classifying data into positive or negative sentiments. The stages of the TF-IDF that have been carried out can be seen in Table 8.  Figure 2 shows the results of the Naïve Bayes classification which shows an accuracy of 97% with negative sentiment results, namely precision has a value of 98%, recall has a value of 98% and f1-score has a value of 98%, while positive sentiment results, namely precision has a value of 94%, recall has a value of 94% and f1-score has a value of 94%. Furthermore, data that has gone through the weighting stage will be classified into two classes, namely the positive class and the negative class with the requirement that if the weight is > 0 then it enters the positive class while the weight > 0 will enter the negative class in the form of a pie chart. The percentage graph in this study can be seen in Figure  3.  Figure 3, it can be seen that the results of sentiment generate more negative sentiment, which is equal to 73.4%, compared to positive sentiment, which is equal to 26.6%. From these results it can be concluded that the community still has an unfavorable view of the SatuSehat application. Therefore, it is hoped that in the future, the SatuSehat application can improve its quality so that users have a positive view of using it. Because a successful application is an application that can make users feel satisfied, useful and make users want to use it again in the future (Lee & Trimi, 2018).

Visualization
Visualization aims to extract information in the form of topics that are often discussed by SatuSehat application users in the Google Playstore application review column.  Figure 4. shows positive wordclouds that are often used by SatuSehat application users to provide reviews on the Google Playstore application. In wordcloud, the larger the word size indicates that the more often these words are used by SatuSehat application users as topics of conversation. Some of the topics that are often discussed by SatuSehat application users are good, "good" means fine, "good" means nice, and "mantap" means excellent.  Figure 5 shows negative word clouds that are often used by SatuSehat application users to provide reviews on the Google Playstore application. Some of the topics that are often discussed by SatuSehat application users in negative reviews are the words update, difficult to mean "difficult", "strange" to mean strange, log in, bugs, and so on.

Conclusion
This study aims to classify user reviews of the SatuSehat application into positive and negative sentiments and assess the performance of the Naïve Bayes method regarding public opinion on the use of the SatuSehat application based on reviews from the Google Playstore application. In addition, this study aims to display a visualization to find the words that appear most frequently in user reviews with the data obtained as many as 25,000. The results obtained in this study are that using the Naïve Bayes method is able to provide high accuracy results, namely 97% with negative sentiment results, namely precision has a value of 98%, recall has a value of 98% and f1-score has a value of 98%, while sentiment results are positive, namely precision has a value of 94%, recall has a value of 94% and f1-score has a value of 94%. Furthermore, the results of the percentage chart show that sentiment results generate more negative sentiment, with 73.4% compared to positive sentiment, which is equal to 26.6%. Therefore, the results of this study can be a recommendation for the company to improve the quality of the SatuSehat application so that it is better in the future.