Research On Compression Strategy And Compression Algorithm Of Historical Data In Process Industry

With the technological advances of modern industrial control and computer technology, supervisory control and data acquisition (SCADA), which reduces the complexity in industrial control, has well developed and has a wide range of applications. SCADA improves work efficiency of automation engineering greatly. As an important component of SCADA, historical database saves important data in process industry field day and night, so it provides first-hand information for management analysis of processing failures. In addition, historical database plays a great part in trend analysis, event processing, report printing and other services.As a result of the increasingly stringent requirements of safe operation and efficient production in process industries and continuous expansion of scale of the project, the amount of historical data sharply rises. To ensure the vast amounts of data can be real-time stored, to reduce storage costs and to increase storage efficiency as far as possible, the data compression is essential need.A strategy of multi-stage compression is proposed for the process data compression in this paper, including the strong noise data smoothing, different recording methods for different signal with actual characteristics, data filtering algorithm for timestamp, switch type, integer type and analog type and streamlining compression of data type of char and float.An automatic parameter control SDT algorithm is presented for the analog data compression in this paper. By adjusting the key parameter in process of data compression, compression error is controlled in the range of permission, so the accuracy is well ensured and the compression ratio is greatly improved. Compared to SDT, compression ratio increases by 100%⁵00% in average in simulation, increases by 61% to 108% in industrial field test. Besides, the absolute difference between actual error and the expected error can limit on 10^-3 orders of magnitude. The algorithm solves the algorithm parameter setting problem in SDT, and avoids the poor compression performance caused by unreasonable parameter settings, so it has important engineering significance.The validity of the strategy of multi-stage compression and the improved SDT algorithm proposed in this paper has been proved by testing and running on the UWnTek platform. The performance of historical database is stable, and data acquisition and data compression run simultaneously.