Research On Fault Detection Technology For Big Data Storage System

With the rapid development of information society,human society is entering the information age of big data;now people in their daily lives all the time in the manufacture of a large amount of digital information,storage of data has shown explosive growth trend,therefore,the current big data storage and management technology has become one of the key issues in the field of information science to be solved urgently.Traditional network storage system uses centralized storage mode,unable to meet the needs of big data mass storage and management,it is currently being replaced by a distributed storage system.Distributed storage system comprises a plurality of inexpensive storage nodes,which capacity,parallel,scalability,reflects the superior performance,has become the preferred storage structure of big data storage system.However,the distributed system with the continuous expansion of the scale,its complexity has continued to grow,although its storage node is cheap,but reliability is not high,prone to failure with disastrous consequences.Because each node storage system failure could lead to paralysis of the whole system and cause significant losses,and so it must be the fault detection data disaster recovery technology makes it possible to take corresponding measures to save storage device before failure to avoid disaster consequences.Fault detection plays "scout" role in disaster recovery technology in the data storage system,high accuracy and timely fast fault detection technology is a necessary prerequisite and effective protection for implementing distributed data storage system disaster recovery;However,traditional adaptive failure detector are used timeout mechanisms,and requested data samples have normal distribution,and the efficiency and accuracy of prediction are still some limitations for high dynamic applications for distributed systems.Based on this,for big data storage systems,fault detection techniques and methods for a more systematic analysis and research,mainly completed the following work:(1)Study the reliability and fault detection technology of a distributed storage system,determine the failure prediction-based,to improve the reliability of big data storage systems of the overall program.(2)The use of BP artificial neural network algorithm for establishing a fault detection model.Estimating dynamic arrival time of the heartbeat information by BP artificial neural network algorithm,so as to solve the sample data must obey the normal distribution problems.(3)Proposed a "ABC-BP fault detection model",which adapt to big data storage systems.Using Artificial Bee Colony Algorithm(referred to as the ABC algorithm)to optimize the BP artificial neural network prediction algorithm,to a certain extent on its own to solve the shortcomings of slow convergence and obtained partial solution,further optimizing the fault detection model.(4)According to the execution environment of the application and the current network status,adjust the heartbeat information security time margin dynamically,further improve the accuracy of failure detector adaptive performance and results.(5)To simulate the implementation process of the ABC-BP fault detection model,assess the feasibility and performance of the model.Experimental results show: Compared to traditional fault detection model,reliability and availability of ABC-BP fault detection models in big data storage systems has improved significantly.