Research On Parallelization Technology Of Remote Sensing Image Classification Algorithm Based On Cloud Computing Platform

Hyperspectral remote sensing image contains a wealth of spatial information and spectral information,which makes it more superior to other remote sensing image in culture recognition and classification,so it has been widely used in military reconnaissance,resource exploration and environmental monitoring.Hyperspectral remote sensing image classification is one of the important contents of hyperspectral image processing.As the hyperspectral remote sensing images have high-dimensional,multi-band,large amount of data characteristics,the existing algorithms of serial classification have high computational complexity and the real-time performance of the algorithm is not satisfied.At the same time,the resolution of the remote sensor continues increasing,which caused the data volume of hyperspectral remote sensing images increase exponentially,while the existing standalone computing platforms can't handle the massive hyperspectral remote sensing image data.Because of the distributed storage and distributed computing characteristics,cloud computing technology can solve the hyperspectral remote sensing image classification problem effectively.The task scheduling algorithm in cloud computing platform impects has a very important impact on the execution performance of tasks.The appropriate scheduling strategy can improve the execution speed of task.Therefore,the parallelization and task scheduling of hyperspectral remote sensing image classification algorithm SCSRC in cloud computing platform are studied,which based on spatial correlation regularization sparse representation.The main works are as follows:(1)Traditional classification algorithms of conventional remote sensing images can't satisfy the classification requirements of large data volumes for hyperspectral remote sensing images,therefore,sparse representation classification method named SCSRC is proposed,which based on spatial correlation regularization.It not only utilizes the spectral information of hyperspectral remote sensing image,but also adds the information from adjacent data in image space,and achieved good result.The implement of SCSRC method on a single machine is studied firstly,then the time performance of this method by experiments is analyzed,which provides the basis for the parallel research in the following cloud computing platform.(2)As SCSRC algorithm has high computational complexity with the restriction of single machine,the performance bottleneck of SCSRC algorithm is analyzed,the SCSRC parallelization methods named MR_SCSRC and SK_SCSRC are designed under Hadoop and Spark platform respectively.The MapReduce method of matrix multiplication based on outer product method is designed firstly in MR_SCSRC,then the algorithm is optimized from three aspects:reducing the number of 10 phases in Map phase,merging calculation logic and implementing localization combine.Spark platform is more suitable for iterative computing contrast to Hadoop.Accroding the realization of MR SCSRC,SK_SCSRC method based on Spark RDD programming model is further designed.Finally,the speed ratio and the spreading ratio of SK_SCSRC algorithm are given by experiments,and the time performance of MR SCSRC and SK SCSRC is compared.(3)The existing task scheduling algorithms of Hadoop consider data locality simply,while ignoring the load balance of the cluster,therefore,based on the genetic algorithm,the task scheduler StaticGAtaskScheduler which can balance the job completion time and cluster load balance is designed and realized.The scheduling schedule verification of StaticGAtaskScheduler on Hadoop platform is given in detail.Finally,StaticGAtaskScheduler is used to verify the performance improvement of MR_SCSRC algorithm.