Research Of Outlier Detection And Data Recovery Based On Statistical Method

As the complicated industry process,the growing demand for the industrial enterprises and the development of control theory and computer technology,soft measurement technology as a kind of low cost, obvious effects of method,has become one of research hotspots in the field of modern industrial control.The soft measurement modeling accuracy is affected by the modeling data,the input data of the soft sensing model are from the industrial field.But it is well known there are many factors in the process of industrial production,such as noise, sensor fault or negligence of the staff,both of these will make some of the data collected abnormal,so the detection and recovery of modeling data becomes very important.Theses mainly from the following aspects were studied.For single variable data(one dimensional data),adopt the method of wavelet transform for testing,through scaling and translation operation functions such as the detailed analysis of data,because the time-frequency characteristics of wavelet transform,can accurately position the outlier.It can extract feature information of the signal effectively,and it can find the location of outliers.for high dimensional data, using principal component analysis method to deal with data,builds the T2 statistics figure and Q statistics respectively.Then, according to each variable’s contribution to the T2 statistics figure to more detailed division of Q statistical figure,selection of T2 statistic figure significantly related variables of statistical figure joint detection,the gross error in checking amplitude smaller strengthened. For industrial data before fill to recovery some of the shortcomings, this paper applies the method of deep learning to the recovery of abnormal data,first, extracting the multi-level feature fromthe data of single variable time series,then,using the BP neural network to train the characteristic variables,recovery the outlier from the nonlinear point of view.Taking the field data collected by PTA industrial process as an example,using the wavelet analysis and principal component analysis to detect the abnormal data of the time point and time period,and then making use of the deep learning method to fill up and recovery the abnormal data,the simulation results show that the detection and recovery effects are perfect.