Research On Detection And Calibration Method Of Abnormal Data In Agricultural Internet Of Things

Agricultural Internet of Things has become one of the most important data sources of agricultural big data.However,the equipment agricultural Internet of Things is affected by manufacturing technology,process and cost,and network transmission,and abnormal data inevitably exists during the data collection process.The existence of abnormal data leads to the deterioration of data quality,which cannot guarantee the intelligent regulation of Io T devices and the effective analysis of data.In order to improve the quality of data collection,this paper focuses on the two main problems of abnormal data and drift in the process of agricultural situation data collection,such as air temperature,CO₂ volume fraction and soil temperature.Research on methods such as online detection of abnormal data,drift data augmentation and drift calibration provides a reference for improving the quality of agricultural data collection.The main work and results of this paper include:1)Online detection method of abnormal agricultural data based on predictive model.An online detection framework for the abnormal agricultural data is constructed based on the sliding window and the prediction models,which including support vector regression,K-nearest neighbor,gradient boosting regression and random forest.The calculation method of the sliding window size is proposed based on data features.The applicability of the prediction models is evaluated by using entropy weight TOPSIS.Through the sheepfold’s monitoring data of the air temperature,the relative humidity,and the CO2 and H2S volume fractions,it is demonstrated that the proposed calculation method of sliding window size is superior to the calculation method simply based on the sampling interval and characteristic period.The prediction errors of these models are negatively correlated with the abnormal detection performance and could impose significant influence on false positive rate.Support vector regression model is the most appropriate candidate for detecting the abnormal data in air temperature and relative humidity whereas the most appropriate candidates for dealing with CO₂ and H₂S volume fractions are gradient boosting regression model and K nearest neighbor model.2)Sample Construction method of Agricultural data based on data augmentation.Firstly,the detection method of abnormal agricultural data is used to preprocess the original data,and then the inverse distance weighting method is used to expand the sensor data and Gaussian random process to construct time and space independent data samples.the sensor data drift process is constructed by using the non-stationary random walk process with linear and other trend terms.The drift simulation is carried out by setting the drift probability threshold of the sensor and determining the maximum drift range based on data characteristics.Finally,the random cutting method is used to realize the construction and augmentation of drift-free data and drift data samples,which provide a data basis for dealing with the problem of agricultural data drift calibration.3)Agricultural data drift calibration method based on deep learning.The sensor drift calibration framework is constructed by squeeze-excitation residual network and multi-scale residual network,and the data set is constructed by soil temperature data.the effects of sample length,drift trend,drift amplitude and spatio-temporal correlation of samples on the calibration capability of the model are analyzed.The results show that,for the samples with spatio-temporal correlation,the squeeze-excitation residual network has better calibration capability than the traditional residual network.The drift calibration capability of multi-scale residual network is generally lower than that of squeeze-excitation residual network and fluctuates due to the increase of sample drift.For the samples without spatio-temporal correlation,the calibration capability of the squeeze-excitation residual network is significantly lower or even lower than that of the multi-scale residual network.