Design And Implementation Of Distributed Time Series Database System For Edge Cloud Scenarios

With the advent of the new infrastructure era,the massive amount of time series data generated by an endless stream of Io T devices has brought great challenges to applications based on traditional cloud computing.In response to this challenge,more and more applications are designed for edge cloud scenarios.In this way,the edge cloud layer is close to various terminal devices and is responsible for full storage of the collected time series data.Then,the data processed by the edge cloud layer is aggregated to the central cloud layer on demand,which can not only reduce the pressure of network transmission,but also reduce the load of the central cloud layer.For time series data storage,most of the existing time series databases only provide general data read and write services and cannot be combined with edge scenarios.Therefore,this thesis analyzes the requirements that the time series database should meet in the edge cloud scenario,and draws the conclusion that the system should meet the functions of distributed cluster,cloud-edge data synchronization,data compression,abnormal monitoring and alarm,and accordingly design and realize a distributed time series database system in the edge cloud scenario.In view of the fact that the standalone version of the time series database is difficult to meet the security,availability and data persistence problems of the system in the edge complex environment,this thesis firstly implements the distributed time series database module based on the open source standalone Influx DB.It mainly includes Meta cluster and Data cluster.The former uses the Raft protocol to achieve a strongly consistent metadata management center;the latter uses mechanisms such as sharding,replicas,and asynchronous data transmission to achieve an eventually consistent data storage center.Secondly,a data synchronization module between the edge cloud and the central cloud is provided on top of the distributed time series database.This thesis achieves the purpose of synchronizing data on demand by providing a custom interface.At the same time,the synchronization interfaces of multi-target databases such as Influx DB,My SQL,and Hive are implemented to adapt to different storage engines in the central cloud.Then,in view of the characteristics of massive time series data and low amount of information per unit density,this thesis uses the spinning door transformation algorithm to design and implement a compression module,which compresses the collected data and stores it,thus improving the storage efficiency of the system without losing effective information.At the same time,the system also uses the threshold filtering method to filter out the collected abnormal data and notify the relevant personnel of the alarm information through the alarm trigger mechanism.Finally,functional test and basic performance test are carried out for the completed system.The test results show that all functions are in line with the expected design and can meet the application of most Io T scenarios.