The Study Of Meteorological Data Acquisition And Data Dining Platform Based On Hadoop

With the development of the information society, our country has entered a stage of rapid development of meteorological undertaking, and meteorological data generated every day are on the increase. The historical meteorological data accumulated can be described as "massive". However, in face of such a mass of meteorological data, traditional method of data processing has been unable to meet the requirements of the current age of big data. How to store and calculate the vast amounts of meteorological data safely and efficiently and dig up some meaningful information fast and accurately have been important issues in the field of data mining in meteorology.This issue develops out of “Tianjin Binhai meteorological early warning platform” and aims to establish a platform integrated meteorological data acquisition, data storage, data mining and application. The major work in this paper can be described as follows:(1) Design the meteorological data acquisition hardware circuit. On basis of understanding the requirements of temperature, humidity, wind speed, wind direction and precipitation measurement of automatic weather stations, sensors are chosed in view of these weather elements, PIC18F8722 is used as the central control chip, and GPRS module is used to transmit data through the wireless network.(2) Study the storage and data mining technology of meteorological data. First, related knowledge of data mining is introduced, focusing on the study of CLARA which is one of the clustering algorithms and suitable for applications of the big data. Secondly, related contents of the open source cloud platform Hadoop is introduced, and the core components of Hadoop — HDFS and Map Reduce are studied. Based on that, architecture is put forward to establish the meteorological data warehouse, and each part is described in detail. Finally, experimental environment was set up, and the CLARA algorithm is transplanted to the cloud computing platform—Hadoop. the experiment is used to verify the efficiency and reliability of the algorithm.(3) Research the application of meteorological data. First, study the basic knowledge of Delauday triangulation. Secondly, on basis of understanding the grid sequence isoline generation algorithm, based on Delauday triangulation, combined the Delauday triangulation, a new isoline generation algorithm is formed. Finally, the experiment is done to verify the advantage of the speed of rendering isoline of the algorithm.