Research Of Distributed Storage Of Massive Remote Sensing Data Base On HDFS

With the rapid development of technology about Global Earth Observation, the Image Data is growing exponentially. At the same time,our country has carried out a series of special and basic research projects,such as high-resolution Earth observation system and so on,following these projects developing,there are a lot of high-resolution remote sensing data, the traditional techniques to handle the mass remote sensing data are becoming more difficult.so people are researching the technology of storage about the super-large scale image data.In the next few years,how to quickly and efficiently access of massive remote sensing data and its management is the an important topicThis paper researched how to quickly and efficiently carry out the massive remote sensing data storage and management technology, choosing the hadoop distributed file system HDFS as the storage platform,comparing the other mainstream about remote sensing data storage solutions in HDFS file system,on that basis,introducing a number of other fine mechanisms,so that it can be applied to storage the massive remote sensing data.Main research contents including:(A) Analyzing the technology about the traditional remote sensing image data storage,and exploring the deficiencies of traditional remote sensing image data storage in face of the rapid development of data size and diversity。After comparing with present stage mainstream Distributed File System, select the HDFS as the remote sensing data storage management research issue.(B) Describing the traditional method of remote sensing image data storage quad-tree technology, traditional quad-tree algorithm needs to consume a large amount of computing resources, timeliness and efficiency is not very satisfactory.Based on the distributed file system MapReduce algorithm,this paper have proposed the quickly build quad-tree algorithm,and proposed the HDFS file system under the four forks tree building methods and construction strategy.(C) Designing the spatial data storage data model based on the Hbase database,so that it can be applied to HDFS distributed file system;Secondly.in this situation,HDFS only has a single metadata node HDFS NameNode,in this situation,there are the system stability problem,adopting Active-Standby to ensure the system’s fault tolerance;once again introducing the Nagios management plug,monitoring mesh nodes distributed file system performance information,thus ensuring stability of the system;(D) In order to solve the high efficiency of massive data service problems, in reference to the OGC standards. Based on HDFS, this paper design a set of data services.The design of the HDFS file system interface to a set of data services that can timely information feedback system data and system status information.(E) Based on the above idea,This paper design experiment to verify the improvement strategies and methods effectively.The results showed the HDFS distributed file system based on remote sensing data for centralized management for HDFS is high-performance, and data storage model can address the growing ultra-large-scale mass remote sensing data storage management issue.at the same time, I have optimization the performance of the existing system,which are using the HDFS for data storage,these improvements are effective.