Research On Hard Disk Fault Prediction Technology In Massive Data Storage System

With the development of satellite remote sensing technology,the number of satellites is increasing,the field of vision is expanding,the resolution is constantly improving,and the amount of remote sensing data produced is increasing exponentially,which brings great pressure and challenges to the ground data storage system.The remote sensing data on the satellite is continuously transmitted,and the ground application business frequently accesses the storage system,resulting in frequent read and write operations on the hard disk,accelerating the aging and failure of the hard disk.Hard disk failure will seriously affect the performance of the storage system.For the remote sensing satellite data ground storage system,the hard disk fault prediction technology in massive data storage system is studied.The anomaly detection of the hard disk status data is used to predict the hard disk failure and improve the maintenance efficiency of the storage system.The main work of this paper is as follows:A hard disk failure prediction model based on the reconstruction error of autoencoder is proposed.At present,mainstream hard disks have a status monitoring and recording function,which records time-series data of status changes during the use of the hard disk.Hard disk failure can be predicted through anomaly detection of hard disk status data.After analyzing the hard disk state data,it was found that there was a serious imbalance between the positive and negative samples.In order to prevent over-fitting and under-fitting of the model during training,it was decided to use a deep generation model for unsupervised learning to perform anomaly detection on the hard disk state data.The model body is an "encoder-decoder-encoder" structure.The long short-term memory network is used to construct the Autoencoder to reconstruct the sample,and the secondary coding is added to realize the reconstruction of the latent vector.Training only needs to use normal samples,and constrain the model to learn the distribution characteristics of normal samples by reducing the reconstruction error of samples and latent vectors.The model also introduces GAN to enhance the training of the model.During the test,the more stable reconstruction error of the latent feature vector is used as the anomaly scoring standard.The anomaly score of the sample exceeds a certain threshold and is judged as anomaly,and the hard disk corresponding to the sample is predicted to fail soon.The experiment is compared with the traditional unsupervised learning method on the two hard disk data sets,and the model has achieved better hard disk failure prediction performance.A hard disk failure prediction model based on GAN's prediction error is proposed.The LSTM network is used to design the GAN generator.The pre-sequence data of the Hard disk status data sample is taken as the input of the generator,and output the prediction of post-sequence data of the sample,thereby avoiding the problem that the re-mapping in the model based on Autoencoder reconstruction causes too long calculation time.The model uses the fully connected layer to extract the latent vectors of the samples.It only needs to use normal samples for training as well.The training of the model is constrained by reducing the error between the samples and the latent feature vectors,and the error of the latent feature vectors is used as the anomaly scoring criterion during detection.The model was compared with the hard disk failure prediction model based on Autoencoder reconstruction errors on multiple data sets.The results show that the model has significantly improved the training speed under the premise of ensuring certain detection performance.