Hadoop Parallel Processing Research And Application For PSInSAR Remote Sensing Data

With the spatial resolution and spectral resolution of all types of remote sensing instruments being constantly improved, and also with the extension of time, there has been a sharp increase in the amount of remote sensing data related to monitoring of land subsidence.Traditionally, PSInSAR generally use single Shell and Matlab script to process remote sensing data serially. Therefore, traditional PSInSAR is insufficient when faces massive data processing. Due to the limitation like slow operation speed of single CPU and low disk capacity, when faces massive data processing, traditional PSInSAR will have some serious problems like load and process slowly or system crash. Therefore, it is necessary to reform the PSInSAR system and improve its running speed and storage capacity to meet the performance requirements of users.Currently,the industry improves performance of single computer hardware generally in order to process remote sensing data in PSInSAR more effective.This method is inefficiency, costly and it has inflexible expansion, it is unable to realize the long time monitoring and processing of data.Based on the analysis of the existing PSInSAR remote sensing image data system, parallel transformation of the system structure is carried out with Hadoop technology in this paper, the processing speed of the system is improved by optimizing the system structure. The Shell and Matlab scripts which can be parallel processed in PSInSAR are studied, and the parallel transformation of serial scripts are carried out with MapReduce. At the same time, the new distributed image data storage management module is added to improve the reading and writing performance of the system. Based on the analysis of Hadoop source code and the operation of a lot of tasks, the configuration parameters of the operations on the Hadoop platform are obtained to optimize the whole system. Ultimately, we compare the stand-alone version and the cluster version of the system through experiments. The effectiveness and practicality of the system are verified, and the performance of the system can be significantly improved.