Research On The Redundancy Mechanism Of Remote Sensing Image With HDFS

The massive remote sensing image data are used basically in distributed storage mode. Especially inthe high resolution data storage system, we need to provide certain data redundancy technology to ensurethe safety, high availability and completeness of data.At present, there are three kinds of data redundancy technique: completely copy technology, disk arraytechnology and controlling delete code coding redundancy technology adopted in the traditional distributedfile storage system. Completely copy and disk array technology can improve system redundancy, but at thesame time it can also increase the demand for system storage space; rectifying delete redundant codecoding technology can make up for the defect of excessive consumption of storage space, but also increasethe burden of I/O operations. In view of the above three methods, this article combines complete replicationtechnology and controlling delete code coding redundancy technology to solve it.Based on the open-sourceHDFS (Hadoop Distributed File System), this paper uses improved redundancy mechanism instead oforiginal redundancy mechanism to slove the conflict between system storage space and system I/O burden,and can guarantee the system I/O speed, and at the same time reduce the demand for storage space insystem in improving the whole system of redundancy.This article researches mainly on a data redundancymechanism for high resolution remote sensing image and proposes an improved redundancy strategy. Mainwork and contributions are as follows.1. On the foundation of research for massive remote sensing image data storage technology and dataredundancy mechanism, this paper mainly researches the hadoop distributed file system andredundancy technique, analyzes for mass remote sensing image storage replication redundancymechanisms and controlling delete code coding redundancy technology.2. On the basis of copy redundancy mechanism and delete redundant code coding redundancymechanism, this paper propose a improved HDFS redundancy strategy in combining both of them, andputs forward the file to read and write process scheme and the code generated by the system after thecoding block management solution.3. The improved HDFS redundancy strategy has carried on the experimental verification to prove thefeasibility of the proposed improvement program, and it has been applied successfully in high resolution major project (ERSI-DBMS) of massive remote sensing image data stored system.