Research On Spatial Data Mining Based On Hadoop

Spatial data mining is a research method which automatically extracts implicit knowledge and spatial relationships. Spatial data can be both spatial point, line, surface entity data, but also spatial objects with geographic location and attribute。 It has various data types, which makes single spatial data mining technology difficult to meet the needs of practical application. And the data always is huge volume, so the spatial data mining is very time-consuming, and can’t effectively meet the timeliness requirements. Some common spatial data mining techniques includes statistical approach, cluster approach, spatial analysis approach, computational geometry approach, etc.The Voronoi diagram method of computational geometry has a natural advantage in expressing the relationship between spatial proximity, and can be used to solve many spatial data mining problems of spatial point, line, and surface entity data. But currently, the researches of the generator of the Voronoi diagram of lines and polygons are few and low efficiency, so weighted Voronoi diagram of polygons have very important research value.Spatial Cluster analysis is another important technique in the field of spatial data mining, especially the K-Means spatial clustering method, which can deal with spatial objects with geographical location and attribute. However, with the development of the information society, the spatial data grows explosively, but the serial algorithm has low computing efficiency and is difficult to process massive spatial data.For the advantage of Hadoop in dealing with large-scale mass data, this paper mainly designed and implemented Weighted Voronoi diagram for Polygons and K-means space clustering algorithm with MapReduce. The results of our paper are summarized as follows:(1) The related technologies of Hadoop are described, and then the working mechanism of HDFS and MapReduce implementation process are analyzed, providing a theoretical basis and platform environment for subsequent algorithm design.(2) Aiming at planar spatial data topology of complex topological structure, it proposed the algorithm parallelization of weighted Voronoi diagram combined with the polygon boundary extraction ideas, and realized the method on Hadoop MapReduce platform. Finally, it used Weighted Voronoi diagram of polygons to solve spatial objects impact scoping problems.(3) Aiming at spatial data with a double meaning of location and attribute, the paper designed and implemented K-Means spatial clustering parallel algorithm on Hadoop. And use Sina Weibo user data to do clustering analysis. Finally, the visualization of clustering results was implemented by Google Map.