Research On Incomplete Data Imputation In Sensor Networks

Nodes hardware failure, network congestion, and adverse environment issues make the data incomplete inevitable. Incomplete data takes severe challenge to the subsequent process, such as data fusion, storage and data mining, since most of these are for complete data. Therefore, incomplete data processing becomes a major problem we face. Traditional incomplete data imputation algorithms are mainly for statistical data, which make the processing either with high complexity, either with low accuracy, all of the algorithms are difficult to meet high requirement both of accuracy and efficiency of sensor network. Thus these algorithms cannot be directly applied to sensor networks.Taking into account people’s different requirement for different data, the paper proposes an imputation algorithm based on the importance of attributes. With attributes reduction techniques, we divide the attributes of missing data system into two parts:important attributes and unimportant attributes, for important part the algorithm use mahalanobis distance algorithm to accuracy imputation, while unimportant use probability algorithm for imputation, this method ensuring both the accuracy and efficiency. The simulation experiments show that the paper proposed algorithm performs better than traditional algorithms, and it’s an effective incomplete data imputation algorithm.For the flow characteristic of sensor network data, full use of the temporal correlation of sensor networks, the paper also proposes an imputation algorithm based on sliding window for incomplete data flow of sensor network. Based on sliding window technology, the algorithm makes definition of nested window and correlation coefficient graph, which get rid of a large number of irrelevant information, minimizing the amount of calculation, using an incremental analysis of the correlation coefficient to calculate spatial correlation between attributes, then use the improved mahalanobis distance imputation algorithm or simple linear imputation algorithm. The algorithm effectively eliminates a large number of irrelevant data, reducing the amount of computation, Thus the algorithm is an efficient method for handling incomplete data stream. Comparative experiments show that this algorithm can effectively impute incomplete data in sensor networks.