Research On Key Technologies Of Quality Control And Truth Discovery System For Meteorological Data

In recent years,a large number of meteorological automatic observation stations have been established nationwide,and the data volume of meteorological data has exploded.The quality of meteorological observation data not only directly affects the accuracy of weather forecasts and climate prediction,but also has an effect on meteorological and research in related fields.Therefore,in order to ensure the representativeness and accuracy of meteorological data,it is necessary to carry out quality control of meteorological observation data.Traditional QC algorithms usually only perform climatological threshold check of a single meteorological element,station or area boundary value check,site time consistency check,and space consistency check between stations.Currently,it is unable to meet the needs of modern meteorological services.In terms of weather forecast and climate prediction,data mining methods are widely used,but they are used less in the quality control of meteorological observation data.There are many observations on the surface meteorological observatory,such as temperature,air pressure,relative humidity,wind speed,wind direction,and precipitation.There are many kinds of meteorological observation data and there are great differences.Observing stations in different regions may have different observation data on the same object,and observing data from different observing stations on the same object in the same region may also be different.At the same time,due to some inaccuracies in input,lost records,outdated data and other issues,there may also be clashes in meteorological data.Multiple data sources may conflict on the description of the same entity object.How to find the correct information from these conflicting information becomes a problem that needs to be solved urgently.Such a problem is also called the problem of truth discovery.Based on the meteorological data,this paper has conducted a series of research work on quality control and true value discovery:First of all,this paper proposes an element-related meteorological data quality control scheme based on Extreme Learning Machine(ELM).According to meteorological principles,there is indeed a correlation between certain meteorological elements.According to the correlation between different meteorological elements,the use of Extreme Learning Machine,an easy-to-use,effective single hidden-layer feed-forward neural network learning algorithm.First,the grey correlation analysis was used to filter the input elements as the input neurons of the extreme learning machine,and then uses the extreme learning machine to output the estimated values of the target components.The experiment proves that this scheme can not only fit the missing value but also detect the abnormal value of the known weather data,and the quality control sensitivity is high and the effect is good.Second,in order to solve the problem of data conflict in meteorological observation data,Multi-TypeTD method model is proposed in this paper.For heterogeneous types of data in meteorological data,use the unified model truth-finding method to iteratively update truth and source weights,and find the most accurate description of meteorological conflict data generated by multiple data sources.In this model,the weight distribution scheme was used to capture source reliability distributions and use different loss functions to characterize different types of data.Experimental results show that compared with the existing truth value discovery methods,this method has obvious advantages in solving heterogeneous data conflicts.