Research On Quality Control Technology Of Marine Observing Data Based On Machine Learning Algorithms

The correctness of marine observing data directly affects the accuracy of the description of the basic characteristics of the ocean,the analysis of the laws and the management decisions.If the distorted or incorrect deep-sea ocean environmental data are put into the long-term time series ocean database,they can be used to analyze and study the distribution characteristics and change rules of physical ocean phenomena,or directly used in the operational prediction and forecast in the fields of ocean,weather and climate science,Finally,the reliability of prediction will be seriously affected.In order to further realize the intellectualization and accuracy of data quality control,this thesis combines machine learning algorithm,statistics and other methods to study the quality control technology of marine observing data,to ensure the accuracy and effectiveness of marine observing data.Data quality control mainly refers to finding abnormal data from the original data and marking and modifying the abnormal data.This thesis selects various marine observing data obtained from buoys and observation stations for experiments and validation,mainly including wave data,temperature and salinity data observed by buoys,and temperature data measured by Xiao Mai Dao observation stations.Multiple data quality control methods centered on machine learning algorithms are proposed for different ocean observation data.In addition,considering that statistical methods also have certain applicability,this thesis will research in part with statistical methods.The research content of this thesis mainly includes the following parts:(1)Research on anomaly detection model of marine observing data.The marine observing data set is divided into univariate marine observing data and multivariate marine observing data.For univariate buoy observation significant wave height data,the statistical method combined with local test method is used for anomaly detection.For multivariable marine observing data such as temperature and salinity observed by buoys,the self-encoder with data reconstruction function is used for anomaly detection.(2)Research on outlier correction method of marine observing data.Because the marine observing data is a group of time series data,this thesis establishes the marine observing data prediction model with the Long Short Term Memory network Model:LSTM algorithm as the core and the ARIMA algorithm as the core,predicts the detected abnormal data,and replaces the abnormal data with the predicted data to achieve the correction of the abnormal value of the data.(3)The design of intelligent control software interface system for marine observing data quality.Combining the above two parts,a software system is designed using Python and Py Qt5 to integrate the various processes,method selection and result display of data quality control into the visual interface,support human-computer interaction,and provide the full-process operation function of marine observing data quality control.