Power Station Engine Failure Early Warning System Using Thermal Camera

: Power station engines are critical infrastructure components that require constant monitoring to prevent failures and ensure an uninterrupted power supply. This paper proposes a failure early warning system based on a thermal camera using a computer vision approach. The system uses a thermal camera to generate thermal images in a video format, which is then processed by an automated fire detection engine and temperature detection engine. The results of these two subsystems are then used as input for an anomaly detection engine, which predicts the likelihood of engine failure. Based on the results of the experiments, it can be concluded that the YOLOv7 model outperforms Faster R-CNN in detecting fires, achieving a higher mAP score on the one-class dataset. The proposed temperature and anomaly detection system also accurately detected temperature levels and anomalies in thermal images. Furthermore, in the failure time prediction experiment, the Holt-Winters additive method with additive errors, additive trend


Introduction
Power stations are critical infrastructures that play a vital role in providing electricity to support the needs of society (Salite et al., 2021). Power station engines are the heart of the power generation process, and their failure can cause significant economic losses and jeopardize the reliability of the power supply. Early detection of engine failure is crucial to prevent catastrophic consequences and ensure the continuity of the power supply (Majchrzak et al., 2021). The development of automated early warning systems for engine failure detection using thermal cameras and computer vision technologies represents a significant advancement in the field (Soori et al., 2023). These systems bring several advantages over traditional manual inspection methods, including improved efficiency, accuracy, and cost-effectiveness (Hassani & Dackermann, 2023). By leveraging the power of computer vision and machine learning algorithms (Adnan et al., 2021), these systems can identify abnormal heat patterns and other tell-tale signs of potential engine failure that may go unnoticed during manual inspections.
Using thermal cameras in engine failure detection is particularly advantageous (Bhadoriya et al., 2022). It allows for the non-invasive monitoring of engine components and provides high accuracy and precision in temperature measurement. Additionally, thermal imaging can capture data in real time, enabling the system to detect and respond to changes in temperature patterns quickly (Filippini et al., 2020). By generating continuous, real-time data, automated early warning systems can alert operators to potential issues before they escalate into more significant and costly problems (Eash-Gates et al., 2020). Therefore, integrating thermal cameras and computer vision technologies in engine failure detection represents a promising approach to enhancing engine safety, reliability, and efficiency across various industries (Andoga et al., 2019). This paper proposes a power station engine failure early warning system that utilizes thermal cameras and computer vision technologies. The system aims to provide a reliable and automated approach to detecting engine failure in real time, allowing for prompt maintenance and repair (Nunes et al., 2023). We evaluate the performance of the proposed system by conducting experiments on a real-world power station engine, and the results demonstrate the system's effectiveness in detecting engine failure with high accuracy (Dwivedi et al., 2023). The rest of this paper is organized as follows: Section II provides an overview of the proposed system. Section III presents the Automated Fire Detection Engine. Section IV describes the Automated Temperature and Anomaly Detection Engine. Section V presents the Failure Time Prediction Engine. Section VI describes the experimental setup and the results obtained from the experiments. Finally, Section VII concludes the paper and discusses future research directions.

Method
The proposed system uses a thermal camera to generate thermal images in a video format, which is then processed by an automated fire detection engine and temperature detection engine. The results of these two subsystems are then used as input for an anomaly detection engine, which predicts the likelihood of engine failure. The final result is disseminated to users as a mobile notification.

Result and Discussion
The system overview is shown in Figure 1. The proposed power station engine failure early warning system consists of three main engines: the Automated Fire Detection Engine, the Automated Temperature and Anomaly Detection Engine, and the Failure Time Prediction Engine. These engines work together to provide a comprehensive and reliable real-time approach to detecting engine failure.

Automated Temperature and Anomaly Detection Engine
The "Automated Temperature and Anomaly Detection Engine" harnesses the power of the Residual Network with 18 layers, colloquially known as ResNet18. This model has been chosen for its capacity to accurately classify and detect anomalies in temperature states across a range of thermal images. ResNet18's can effectively learn intricate patterns and details in the data through a series of convolutional layers that utilize residual connections, making it adept at identifying subtle changes in temperature states (Alzubaidi et al., 2021).
Moreover, ResNet18's ability to combat the vanishing gradient problem, thanks to its unique residual connections, allows it to perform exceptionally well in deep learning tasks, like anomaly detection in thermal images (Binta Islam et al., 2023). In a series of experiments, the model has consistently proven highly effective, achieving impressive validation accuracy in both multi-class temperature state and anomaly detection tasks. The hyperparameters used for training are set as follows: the optimizer used is Stochastic Gradient Descent (SGD).

Failure Time Prediction Engine
The failure time prediction engine uses the results of the anomaly detection engine as input and predicts the time to failure of the engine. The engine uses machine learning algorithms to learn the patterns of engine failure and then applies the learned models to the current data to predict the time to failure. This research used nine methods for Failure Time Prediction Engine. In this section, we present the experimental results of the proposed Failure Early Warning System. The experiments were designed to evaluate the system's performance in detecting and predicting potential failures in power station engines. Three experiments were conducted, each focusing on different subsystems. The first experiment evaluated the performance of the Automated Fire Detection Engine, the second experiment evaluated the performance of the Automated Temperature and Anomaly Detection Engine, and the third experiment evaluated the performance of the Failure Time Prediction Engine. The experiments showed that the proposed system provides an effective and reliable approach to detecting and predicting potential failures in power station engines, improving their reliability and reducing downtime and associated costs.

Experiment 1 -Automated Fire Detection
Considering the objective of detecting the location of fire objects, we explored the use of suitable deep learning architectures that have been proven to yield high performance, namely YOLO version 7 and Faster R-CNN (Hussain, 2023). Based on previous experience, YOLO has shown excellent performance in various object detection tasks in terms of both accuracy and speed. However, in this study, we also investigated an alternative and comparative architecture, Faster R-CNN, which is generally slower but is considered to be better at detecting small and closely spaced objects or objects with non-standard aspect ratios as compared to YOLO.
In evaluating the trained models, several metrics or measures can be used as a benchmark to assess the quality of the models (Vishwakarma et al., 2021). Each model for different purposes has its own set of metrics (Petter et al., 2008). As previously explained, this study uses two types of models: detection models and classification models. For detection models (Iwendi et al., 2020), the metrics used are Intersection over Union (IoU) (Chen et al., 2022) and mean Average Precision (mAP). The IoU measures the overlap between the predicted bounding box and the ground truth bounding box, while the mAP measures the average precision across different levels of recall (Padilla et al., 2021). These metrics are commonly used in object detection tasks and provide a quantitative evaluation of the performance of the detection model (Hodges et al., 2021). The experiments used two different models, YOLOv7 and Faster R-CNN, and two datasets, one (fire) and threeclass (fire, smoke, others) (Almazroa & Alotaibi, 2023). The results show that the model had the most difficulty detecting the "other" class, with a mAP of only 0.2. The purpose of including the "other" class annotation is to improve the precision of fire and smoke detection, to avoid false positives on similar objects. Therefore, the"Other" class is not a priority for improving accuracy if fire and smoke detection performance are good. The results demonstrate the effectiveness of the proposed approach in detecting fire and smoke objects using deep learning models and highlight the importance of considering the performance of each class in evaluating the model's performance.

Experiment 2 -Temperature and Anomaly Detection
Firstly, we formulated the problem of temperature level detection and anomaly detection as an image classification problem. We then prioritized selecting a lightweight deep-learning architecture for efficient resource utilization and fast detection processing. The model selected for this purpose is the Residual Network with 18 layers, commonly known as ResNet18, which has a model size of 44 MB.
The hyperparameters used for training are set as follows: the optimizer used is Stochastic Gradient Descent (SGD) (Tian et al., 2023) with a learning rate (lr) of 0.001 and momentum of 0.9. The training is conducted for 10 epochs, and the best model is selected based on the best validation accuracy achieved during the training process. Using an SGD optimizer with momentum effectively improves deep learning models' convergence speed and performance. The learning rate of 0.001 ensures stable training and prevents overfitting. The training process is conducted for 10 epochs to ensure the model has sufficient time to learn the dataset's features and achieve optimal performance. The result of experiment 2 is shown in Table 2.  Table 2 presents the results of the validation experiments for the proposed system. The experiments were conducted for three tasks: temperature state classification, temperature. Table 3 Validation experiment results state, anomaly classification, and anomaly detection. The validation accuracy is used as a performance metric for each experiment. The results show that the proposed system achieved 100% validation accuracy in the temperature state classification task (9 classes), demonstrating the effectiveness of the proposed approach in accurately detecting temperature levels. In the temperature state and anomaly classification task (10 classes), the proposed system achieved a validation accuracy of 95.56%, indicating good performance in detecting temperature levels and anomalies in thermal images. In the anomaly detection task (3 classes), the proposed system achieved a validation accuracy of 98.89%, highlighting the effectiveness of the proposed approach in detecting anomalies in thermal images.

Extraction of Temperature Level Sequences from Video
For the next task, which is failure time prediction from thermal video monitoring, the temperature state detection process is applied to a sequence of frames from the video. Figure 2 below illustrates the process from the video to the failure time prediction. Starting with an input video or CCTV streaming, frame sampling is performed based on time intervals, for example, every 5 minutes. Then, temperature state detection is directly applied to each sampled frame (Varshini et al., 2021). Subsequently, a sequence of temperature states is prepared with a length determined by the failure time prediction model, for example, the last 10 data (Petropoulos et al., 2022). After the sequence of temperature states is ready, it is passed to the failure time prediction model, which outputs the estimated time until failure (Van Dinter et al., 2022). Figure 2 illustrates the sequence of steps taken for extracting temperature state sequences from the video for the failure time prediction process. The frame sampling is performed based on the defined time intervals, and the temperature state detection is applied to each sampled frame. The sequence of temperature states is then prepared according to the length determined by the failure time prediction model. Finally, the sequence is fed to the model, which outputs the estimated time until failure Experiment 3 -Failure Time Prediction Firstly, a time series analysis is conducted on the model dataset. 6 time series models can be used for regression analysis on the available dataset. This research compared different time series forecasting models and their associated goodness-of-fit measures, including Akaike Information Criterion (AIC), corrected AIC (AICc), Bayesian Information Criterion (BIC), and R squared. The models include combinations of additive errors, trend, and seasonality components. Figure 3 shows the time series models performance.   Table 4 from the nine prediction models used Table 4 summarizes the performance of nine different prediction models in terms of accuracy (acc.), standard deviation, gains, total time (in ms), training time (1,000 rows), and scoring time (1,000 rows).
All models have similar accuracy and standard deviation values, indicating they are equally effective in predicting anomalies (Nobre et al., 2023). The models differ in total time, training time, and scoring time. The Fast Large Margin model has the shortest training time, while the Gradient Boosted Trees model has the shortest scoring time. The Support Vector Machine model has the longest total time, including training and scoring time. The Decision Tree model stands out in terms of its short training time of only 3.29 ms and its overall performance, which is on par with the other models in terms of accuracy and standard deviation. This information highlights the strengths of the Decision Tree model and may make it a good choice for applications where training time is a critical factor.
In the third experiment, the Holt-Winters additive method with additive errors, additive trend, and additive seasonality model stands out as the best fit among the models evaluated, with the Decision Tree model also showing good performance and a short training time, indicating its potential suitability for applications where training time is a critical factor.
After analyzing the performance of the different models, several areas for future work were identified. Firstly, further investigation could be done to improve the accuracy of the models, especially for tasks that involve thermal images and videos. One approach could be to incorporate more sophisticated feature extraction techniques, such as deep feature extraction, to capture more complex patterns in the data. Secondly, the models could be further optimized to reduce the training and scoring times, especially for larger datasets. One potential solution could be implementing distributed computing techniques to parallelize the training and scoring processes across multiple computing nodes. Thirdly, the models could be extended to incorporate more data types, such as audio and text, to enable more comprehensive anomaly detection and prediction. For example, audio data could detect abnormal sounds, while text data could be used to identify abnormal patterns in written descriptions of equipment behavior.
In conclusion, the results of this study demonstrate the effectiveness of deep learning and machine learning methods for anomaly detection and prediction tasks in industrial settings. Further research and optimization of these methods could lead to even greater accuracy and efficiency in detecting and predicting anomalies, ultimately improving the safety and reliability of industrial systems.

Conclusion
This paper proposes a failure early warning system based on a thermal camera using a computer vision approach. The system uses a thermal camera to generate thermal images in a video format, which is then processed by an automated fire detection engine and temperature detection engine. The results of these two subsystems are then used as input for an anomaly detection engine, which predicts the likelihood of engine failure. Based on the first experiment, YOLOv7 outperforms Faster R-CNN regarding mAP on the oneclass dataset and performs relatively well on the threeclass dataset. Moreover, YOLOv7 is significantly smaller than Faster R-CNN, which may make it a better choice for applications with limited computing resources. The second experiment shows that the proposed system achieves high accuracy in temperature state classification, temperature state and anomaly classification, and anomaly detection tasks, indicating the effectiveness of the proposed approach in accurately detecting temperature levels and anomalies in thermal images. These results demonstrate the effectiveness of different models and approaches for various tasks, highlighting the importance of carefully selecting the most suitable method for a given application.