Research On Vector Data Rendering Technology Based On Hadoop And Mapnik

With the development of earth observation techniques such as global positioning navigation system,satellite remote sensing and aerial reconnaissance,sensor network progress,the means to obtain geospatial data becomes more advanced.GIS got a rapid development in the 1990 s,the characteristic of the geospatial data including large scale,various types and complex structure becomes more pronounced.Geographic information department of surveying and mapping has a real big data, only the data quantity of the western mapping project is 13.4 TB. How to store and process the GIS data efficiently and reasonably,how to improve the map rendering performance to enhance real time service of the spatial data,how to satisfy the new requirement of the government for geospatial information and technical supporting, it is related to the prospects of geographic information department of surveying and mapping directly.Apache’s Hadoop is a data processing platform with high reliability,high efficiency,high fault tolerance and high efficiency,it has been widely used at present,it support data storage and processing with PB level.This paper has carried on the practical research work in depth as following aspects: storing and processing spatial vector data based on the HBase,dividing the spatial vector data based on the MapReduce and distributed rendering spatial vector data,the main work and innovation as follows:(1) Designing the HBase form format to store and query spatial vector data based on the principle and characteristics of Rowkey and Column Family,which belongs to the HBase;(2) Proposing an algorithm "MultiThreadInsertToHBase" which is used for importing spatial vector data to the HBase under the condition of multithreaded and combined with the HBase requirement for input data types,it also realized the algorithm programmatically;(3) This paper proposed an algorithm "MapReduceDivideToShp" which is used for dividing the spatial vector data based on the MapReduce and the GeoTools, combined characteristics is that the MapRedcue could performing distributed compute operation on the HBase,it also realized that result that dividing spatial vector data in the HBase into several smaller vector data files with Shapefile format according to set conditions;(4) Analyzing and discussing internal rendering mechanism and related API details of the open source rendering engine Mapnik, it made the Hadoop experimental cluster could render spatial vector data With a reform. It also realized distributed rendering for the spatial vector data based on the experimental cluster and improved the rendering efficiency.The experiment result shows that storing and processing spatial vector data based on the HBase is completely feasible, the spatial vector data distributed rendering can improve the rendering efficiency of large-scale spatial vector data significantly.