Research On Assessment Model Of Agro-meteorological Disasters Based On Big Data Processing Techniques

With the increasingly close influence of climate environment on agricultural production, how to efficiently deal with the massive growth of meteorological data and improve the scientific nature of agro-meteorological disaster warning is becoming the top research of the agro-meteorological disaster. Meteorological data has the characteristics of huge data, various types, high redundancy, low value density, so the traditional data analysis methods often fail to achieve high processing efficiency. The major works of this paper are about more efficient big data processing technology which is applied to the agro-meteorological disaster warning based on the characteristics of meteorological data, combined with the research results of data analysis.The processing framework of meteorological data and the big data processing technology are firstly summarized. The existing big data preprocessing technology,classification technology and their performance are analyzed. Then, adaptive boosting technology and distributed processing architecture are summarized. Besides,the function of combination classifier and parallelization techniques in big data processing is analyzed. These studies provide preparation for the next step research of agro-meteorological disasters prediction.Aiming at the characteristics of various types and attributes of meteorological data, combined with rough set theory and parallel processing technology, an attribute reduction algorithm of information entropy based on MapReduce is proposed.Repeated and redundant meteorological data are deleted to achieve the compression and refining of knowledge by attribute reduction algorithm of information entropy.And then main task is divided to realize distributed processing by using MapReduce architecture. Simulation result shows that the parallel algorithm has faster processing speed, which is effectively applied to the reduction of meteorological data.Aiming at the problems of inefficient classification about agro-meteorological disasters, a combination classification model of k-Nearest Neighbor based on MapReduce is proposed. The KNN classification algorithm has been fused based on the Adaboost algorithm and each KNN classifier is constructed by finding the optimal number of KNN based classifiers. And then distributed processing of the algorithm is realized by using MapReduce parallel architecture. Simulation result shows that the parallel combination classification model has higher accuracy and faster processing speed.Finally, taking the grade index of low temperature and scant light disaster as the example,agro-meteorological disasters will be classified and predicted by using the big data processing technology which presented in this paper,to realize assessment and early warning of agro-meteorological disasters.