Research On Water Resources Outlier Analysis And Detection Based On Machine Learning

Outlier detection of water supply data is a key link in water resources information processing and decision-making.It is not only a prerequisite for the normal operation of the water resource intake information monitoring system(WRIIMS),but also an important support for promoting the construction and management of water resources system in smart city.With the development of Internet,Internet of Things and big data technologies,the process of informationization of water resources system is being accelerated.At the same time,the problem of data authenticity is becoming more and more serious.The outlier detection of water supply data is facing new challenges.However,the traditional outlier detection mode is usually realized by manual sampling,which leads to low efficiency and high false detection rate and will not adapt to the coming massive data.Therefore,the research of efficient outlier detection algorithms in water supply data has become one of the most important issues.This article mainly focuses on the information processing of water supply data,and the outlier feature analysis and detection methods.The main work is briefly concluded as follows:1)A method of outlier detection and feature analysis based on One Class Support Vector Machine(OC-SVM)is proposed,which can solve the training problem of unbalanced and unlabeled training set for actual water supply data.With the scoring function of OC-SVM,the outlier characteristics can be analyzed.Furthermore,we analyzes and summarizes the characteristics of daily water supply data from three dimensions of one-dimensional,three-dimensional and seven-dimensional,respectively from this experiment.In addition,the characteristics of outliers in the hourly water supply data set are also analyzed from the one-dimensional and twenty-four-dimensional situations.It achieves good results,and provids a reliable basis for the outlier detection of WRIIMS.And this work can also facilitate the design of the later outlier detection model.2)An algorithm of outlier detection based on Joint Auto-Encoder(JAE)network is proposed.In this method,a weight sharing Auto-Encoder network model is designed,and the hybrid loss function and a double thresholds are proposed.This model solves the problems of difficult feature extraction and difficult selection of outlier thresholds in unbalanced daily water supply data.In the experiments,the JAE method is applied to three data sets: synthetic data set,MNIST data set and water supply data.The performance of the model is evaluated by ROC and AUC values,which proves the effectiveness and adaptability of JAE in outlier detection.Meanwhile,the JAE method is compared with AE net,OC-SVM and OC Deep SVDD methods,which shows the superiority of JAE in outlier detection.Finally,through the water resources information fusion and decision support platform,the effectiveness of the JAE outlier detection algorithm is verified,and the JAE model is applied to real-time intelligent monitoring of real water supply data.