Research On Remote Sensing Data Storage Technology Based On Distributed System

With the development of remote sensing in China,remote sensing data is showing an exponential growth trend.In forestry industry,the utilization rate of remote sensing data is increasing year by year,and the effective storage and management of massive remote sensing data has become one of the major challenges facing today.The traditional centralized storage mode based on single node is small in capacity and poor in scalability,so it is difficult to meet the storage requirements of remote sensing data.Distributed storage solves many shortcomings of centralized storage,but the existing distributed storage scheme does not optimize remote sensing data.There still exist such problems as single data format support,waste of storage space,slow indexing speed and so on.This paper studies the technology of distributed storage of remote sensing metadata and image data.A distributed storage model adapted to remote sensing data storage is proposed,and an application oriented remote sensing data storage framework is designed.The experiment shows that the new storage model can effectively adapt to the storage requirements of remote sensing data and improve the storage and access efficiency.(1)Remote sensing metadata storage and access model.In this paper,data governance is carried out,and the concept of metadata set based on classification is put forward to achieve effective coverage of metadata set attributes.A distributed storage model of remote sensing metadata based on HBase is proposed,and the distributed storage of remote sensing metadata is realized.A metadata access model based on density clustering is proposed.And the DBSCAN algorithm is improved to implement the model.(2)Distributed storage optimization of remote sensing image data.On the basis of metadata access model,the image data represented by metadata access model can be divided into high frequency data,low-frequency data and grey data.For high frequency data,three replica redundancy storage strategy is used.For low frequency data,a single replica redundancy storage strategy based on MSR encoding is implemented.The zero copy storage is realized by using SEC encoding for grey data.Based on Peano coding,an improved image data indexing method is proposed,and the parallel index of image data is realized by using Spark.The HBase storage model is established,and MapReduce is used to parallelize the import.(3)Application oriented remote sensing data storage framework.Unmodified data access is implemented on the existing platform.The data read interface of distributed storage is realized by GDAL,and SuperMap oriented HDFS/MongoDB bidirectional data conversion interface is designed,which meets the engineering application requirement of distributed storage model.