Research On Cleaning And Compression Method Of Industrial Perceptual Data Based On Elastic Edge Computing

With the rapid development of Industry 4.0 and the Internet of Things,the number of access devices of the industrial Internet of Things has increased rapidly,resulting in the emergence of massive industrial perception data.However,the industrial cloud based on cloud computing is difficult to meet the low-latency feedback requirements of massive data processing.Compared with centralized cloud computing,the computing services provided by edge computing are closer to the terminal devices.Although the resources of edge computing are very poor compared with cloud computing,its response speed is faster because it is close to the devices.Uploading massive data to the cloud for processing increases bandwidth pressure and prolongs transmission time.Therefore,aiming at the above problems,this paper takes edge computing as the research point,makes use of its fast response speed,builds an elastic edge cluster at the edge layer and designs a reasonable resource allocation method.By studying the cleaning and compression method of industrial perception data based on edge computing,it can reduce the burden of cloud center and reduce the communication cost.The main research contents are as follows:(1)Research on the construction of edge cluster and resource elastic allocation method.Because the computing resources and storage resources of a single edge node are limited,it is difficult to achieve high efficiency in analyzing and processing massive data.Therefore,the edge cluster based on k3 s is studied in order to make effective use of idle edge resources,and the method of resource elastic allocation is studied on the cluster to form an elastic edge computing mode.The main edge node dynamically allocates the computing tasks to other nearby edge nodes according to the demand,and utilizes the computing resources of multiple nodes to process the computing tasks quickly and efficiently,which lays the foundation for the subsequent research on data processing methods of adaptive nodes.Finally,a cluster is built,and the effectiveness of the allocation method is verified by experiments.(2)Research on perceptual data cleaning method based on isolated-random forest.Because the sensor is affected by the environment and its own performance,the quality of the mass sensing data collected is low.Therefore,analyzing the characteristics of the massive sensory data in the industrial field,a sensory data cleaning method based on isolation-random forest is proposed,which uses the isolation forest to detect outliers in the data,and then uses the improved random forest to take the outliers as missing values and predict together with other missing values.Finally,the cleaning algorithm is made into images,which are convenient for deployment and scheduling on the edge cluster.Through algorithm comparison experiments,the cleaning effect of the proposed method on different types of data is verified.(3)Research on lossless compression method of perceptual data based on entropy reduction transform.Aiming at the problems of high bandwidth occupation and high time delay caused by long-distance transmission of massive data,in the case of effectively cleaning the data,the theoretical limit of lossless data compression is studied,and a lossless compression method of entropy reduction transformation based on optimal difference and linear fitting of difference and entropy reduction transformation is proposed.According to the frequency and fluctuation trend of data collection,different entropy reduction methods are adopted to increase the data compressibility,and LZO(Lempel-Ziv-Oberhumer)is selected as the secondary coding compression method.The compression algorithm is also made into a image and deployed on the edge cluster.Through the algorithm analysis and comparison,the performance of the compression algorithm in this paper is verified.