The Research And Implementation Of Remote Sensing Data Storage And Management System Based On Hadoop

With the development of satellite remote sensing techniques, the size of remote sensing data is growing rapidly, which makes them difficult to store and manage with traditional methods. Recently, cloud computing has provided a solution to process remote sensing data. In this thesis, cloud computing technology was applied to the GIS field, and research was practiced to implement a remote sensing data storage and management system based on Hadoop. The main work is listed as follows:(1) Design a remote sensing data storage and management system based on a Hadoop cloud computing platform. The main functions include fast storage, parallel pyramid build, and on-demand publication for remote sensing data.(2) Design and implement a module for storing remote sensing data rapidly. This module is responsible for downloading remote sensing data by supporting distributed multi-threaded and broken-point continuously-transferring download based on the HTTP and FTP protocols. It is capable of storing large-scale remote sensing data. The storage nodes can be extended according to the actual system requirements, making it convenient for the remote sensing data management.(3) Design and implement a module for the parallel pyramid build of remote sensing data. Based on a MapReduce's remote sensing image pyramid parallel building algorithm, this module implements layered cutting and tiling storage for large-scale remote sensing data. It uses the open source library GDAL with quick access to the raster data, which provides data source for parallel cutting.(4) Design and implement a module for on-demand publishing remote sensing data. This module is responsible for user accesses to remote sensing data. It adopts GeoWebCache as the open source tile map service middleware, and HBase for tile storage, thereby being capable of dealing with a large number of user concurrent accesses, e.g. map loading or dragging.(5) Experimental results show that the proposed method in this thesis is reliable and practical. Our remote sensing data storage and management system based on the Hadoop platform processes remote sensing data effectively, hence improving user experiences in some extent.