Research On Outlier Detection Method For High-dimensional Sensor Data

With the rapid development of sensor technology,wireless sensor networks are increasingly being used in industry,agriculture,medical,health and other industries.The sensor nodes are scattered in the target area,and the environmental parameters(temperature,humidity,CO2 concentration,etc.)in the target area can be collected by wireless sensor nodes,which can monitor the internal environmental changes in the scattered area in real time.In order to detect unexpected events in natural environment timely and accurately,monitor the health status of sensor networks and improve the reliability of wireless sensor networks,outlier detection of data collected by sensors is particularly important.The characteristics of sensor data are mainly high-dimensional.Unlike the processing of low-dimensional data,the processing of high-dimensional data will generate greater space and time complexity.In this paper,four sets of real high-dimensional data sets from machine learning UCI database and a set of synthetic high-dimensional data sets are used as the research objects of high-dimensional data.By introducing two data dimension reduction models(DBN and PCA models),the data set is mapped from high-dimensional space to low-dimensional space,and then outlier detection of low-dimensional spatial data is realized by combining the concept of classification model in machine learning.So the data can be divided into normal and outlier categories by classification model.This method can not only reduce the space and time complexity of the algorithm,but also reduce the rate of outlier detection.The main work of this paper are as follows:1.A high-dimensional data outlier detection method based on two different data dimension reduction models(DBN and PCA models)is proposed.This method uses Deep Belief Network(DBN)and Principle Component Analysis(PCA)to reduce the dimension of high-dimensional data,and then inputs the dimension-reduced data into Quarter-sphere Support Vector Machine(QSSVM)model for outlier detection.The method consists of four steps:the first step is to use DBN model to reduce the dimension of high-dimensional data,mapping data from high-dimensional space to low-dimensional space,and realize feature extraction of high-dimensional data sets;the second step is to normalize data,normalize the data from 0 to 1 and centralize the data set to the origin-centered position;the third step is to improve the QSSVM model,which converts the linear optimization problem into the sorting problem,so as to reduce the computational complexity of the algorithm in space and time.The fourth step is to input the processed data into the improved QSSVM model for real-time outlier detection.By dividing the data into normal and outlier categories,outlier detection can be realized.2.Evaluate the performance of the proposed method.Four sets of real data and one set of synthetic data sets,are used for simulation experiments on MATLAB platform.The performance of outlier detection of the proposed method is evaluated by different metrics,and compared with the existing methods under these metrics.The experimental results show that compared with DBN2-OCSVM,DBN2-QSSVM reduces the computation time by more than 90%and improves the accuracy of outlier detection by more than 6%.Similarly,PCA-QSSVM reduces the computation time by more than 90%and improves the accuracy of outlier detection by more than 5%compared with PCA-OCSVM.