Detection And Filling Of Temperature Anomaly Based On LSTM Model

Climate issues affect not only people's daily life but other social and economic development as well.Meteorological monitoring data is the business basis for studying weather and climate change issues.Accurate and high-quality meteorological monitoring data can provide more reliable information for meteorological analysis and meteorological academic research.As automated measurement technology develops,the data volume of meteorological monitoring data increases unceasingly.The importance of meteorological monitoring data is self-evident.In the process of data storage and transmission,the abnormal data,which is caused by various uncontrollable factors,is appeared in the meteorological monitoring data.The meteorological monitoring data is incomplete and unreliability.Therefore,the detection and filling of abnormal data in incomplete and inaccurate meteorological monitoring data sets,which adopt by a reasonable and effective way become the top priority in meteorological research.The temperature data,which is an important element in meteorological monitoring data,is closely related to meteorology and widely used in life.For that,temperature data is suitable as a breakthrough point to study data anomaly detection.In this thesis,the change characteristics of temperature data is systematic studied.For deeply understand the detection and measures to fill out abnormal values of temperature data,the causes of abnormalities in the process of measurement,transmission and storage are investigated.Previous researches about the quality control methods of the abnormal values of temperature data are retrieved and analyzed.Therefore,temperature data is a kind of time series data,which is the basis of the research of this thesis.The basic model is the LSTM(Long Short Term Memory Neural Network)model.Based on the temporal continuity of temperature data and the correlation of spatial position,the topology of the standard LSTM model is improved upon requirements.For the time continuity,a new type of network structure with LSTM models,which is named as Extra Double Input Double Hidden LSTM(EDIDEHLSTM model),is introduced.Based on the EDIDH-LSTM model and the correlation of spatial position,Time and Space Superimposed LSTM model(TSS-LSTM model)is proposed.The two models proposed are verified by the designed experiments respectively to detect and fill the anomaly data in the temperature data set.The main work in this thesis can be summarized as the following two parts:(1)Signing a two-layer LSTM model structure with additional data input and two hidden layers,for short the EDIDH-LSTM model.And designing a missing value processing algorithm to fill the missing temperature value to correct the temporal continuity of the temperature time series data set,in order to keep the continuity in time.The precipitation factor is used as an additional input,and combining with the temperature data trained by the first layer of the LSTM hidden layer,the second layer of the hidden layer training model with the LSTM unit is passed.By experiments with different iterations and optimization methods,the appropriate number of iterations and optimization methods are finally selected.The is thereby built by improve the activation function.In order to verify the validity of the model,experiments were carried out with different data sets,and compared with the standard LSTM model and the sine correction model GM(1,1).The final experimental results show that the proposed EDITH-LSTM model is in the data set.Above all,the abnormal value of the temperature data can be effectively detected and filled,which has better precision than the other two models.(2)Based on the previous model,a LSTM model with spatio-temporal characteristics,Time and Space Superimposed LSTM(TSS-LSTM),is designed for the target site.For the characteristics of the correlation of temperature data in spatial position and the problems in the experiment of the previous model EDID-LSTM model,the missing value processing algorithm is improved to make it more consistent with the temporal variation of temperature data.The abnormal data in the sequence increases the time series data(possibly with outliers)of the simultaneous phase of the auxiliary site as input.The appropriate number of iterations and optimization methods is selected and the activation function for model building is improved.Experiments with the temperature data and geographic factor data of four different meteorological monitoring stations were used for verifying the validity of the model.The spatial position correlation and time series similarity were compared and mutually corroborated with the EDIDH-LSTM model.The experimental results show the availability of the model and the characteristics of the temperature data to a certain extent,which is helpful for the research and detection of abnormal values of temperature data.The possible innovations of this thesis introduce in the following aspects: the abnormal value detection algorithms based on time series are mainly studied in medical treat,industrial fault,aircraft,etc.on the contrary.There are relatively few studies on temperature data,and there are few methods for detecting and filling outliers.Traditional methods only study the processing of temperature abnormal value from a single time or space perspective;there is no uniform and effective standard algorithm for the processing of missing values.For different scenarios to detect temperature outliers,outliers of a single site are not only detected,but also detected by combining data of multiple sites.In order to optimize the model better,different experiments were carried out on optimization algorithm and activation function.