Research On Storage Of SCADA Historical Data Based On Markov Model

In the era of big data,the data generated in industry communities increases by geometric series.It is essential to efficiently store the data collected by the sensor in the SCADA(Supervisory Control And Data Acquisition)system.Historical data as an important part of the SCADA system,which can effectively ensure system stability and normal operation.Before the emergence of NoSQL(Not Only SQL),relational databases are the primary means of historical data storage.However,in the face of massive historical data storage,relational database has been unable to meet the requirments of high concurrency,high real-time and high reliability of SCADA system.It is of great significance to study the storage of SCADA historical data in this context.This dissertation mainly focuses on the data compression algorithm and Markov model to launch the research work,including the improved SDT(Swinging Door Trending)algorithm is applied to the data preprocessing of SCADA historical data storage.In addition,the MongoDB data load balancing algorithm is optimized and improved based on Markov model.The main research contents include the following aspects.(1)MongoDB cluster storage scheme based on replica set+sharding is proposed aiming at the shortage of relational database in mass data storage.(2)The data compression strategy is proposed according to the characteristics of SCADA historical data,and the SDT algorithm used for compression of analog quantity is studied especially.In view of the insufficiency of the SDT algorithm,an improved algorithm based on sine curve is proposed and applied to the data preprocessing before the storage of SCADA historical data.(3)Three kinds of MongoDB cluster structures are studied,including master-slave cluster,sharding cluster and replica set+sharding cluster.In order to meet the requirements of mass storage and dynamic expansion,this dissertation proposes the overall framework of replica set+sharding cluster.(4)The automatic fragmentation mechanism of MongoDB is thoroughly studied and a solution based on Markov model aiming at the shortage of data balance.It improved the data load balancing algorithm by using the characteristics of Markov's post invalid and probability transfer.And it is proved by experiments that the improved data load balancing algorithm can reduce the transfer of chunk and improve the read and write performance of MongoDB cluster.