The Generalized Model And Parallel Computing Methods For Pixel-level Remote Sensing Image Fusion

In general, the panchromatic and multispectral images covering the same area are acquired by satellite. Using image fusion technology, the structural and textural details of the lower resolution multispectral image are enhanced by adopting the higher resolution panchromatic image, thus a higher resolution image with spectral contents is generated. Image fusion is an important step in the processing workflow of satellite imagery, which is extensively applied to visual interpretation, automatic classification, information extraction, feature enhancement, display of3D scene, generation of ortho-rectified image, change detection, and image based mapping. With the increase in the resolution of satellite imagery and in the number of on-orbit satellites, it is a great challenge for image fusion algorithms, which are usually data-and computation-intensive, to process a large amount of data in a short period of time.Although some algorithms were presented in terms of satellite image fusion, there are still several existing problems. The existing models for image fusion usually depict one or two categories of image fusion techniques and are not general. A more general model which can characterize most commonly used fusion algorithms is lack. High performance computing coupled with image fusion algorithms is not very general, and there do not exist any methods of parallel computing which can be applied to most algorithms and can largely promote the processing performance of image fusion algorithms. Most fusion algorithms can not adjust between maintenance of spectral characteristics and enhancement of spatial details, and few algorithms with the capacity of adjustment have many parameters for adjustments and demand a large amount of computation resulting in inconvenient use of these algorithms. The research field still lack fusion algorithms which can adjust fusion effects with few parameters for adjustments and can easily be parallelized.In the light of the above problems, three innovative studies are as follows.(1) A generalized model for satellite image fusionOn the basis of the analysis and mathematical deduction for three general categories, a generalized model for pixel level image fusion is presented. The model can express the relationship between the sharpened higher resolution multispectral image and the original multispectral image, the spatial details extracted from the higher resolution panchromatic image and the adopted fusion strategies by a simple mathematical formula. The calculation expressions of two important variables in the generalized model for commonly used algorithms are listed. Using the generalized model, similarities and differences between these algorithms can be analysed. When the generalized model is used to implement and execute fusion algorithms, the basic steps corresponding to each fusion algorithm are similar, which is useful for software modules’ integration and re-use. For some fusion algorithms, it can discard certain unnecessary steps, thus reducing computational costs.(2) A parallel computing method coupled with the generalized model on a multi-core computerA parallel computing paradigm, which can be applied to implementation of most algorithms and can achieve high performance, is presented for image fusion algorithms combining the generalized model and a multi-core computer. The parallel framework into which eight typical fusion algorithms are integrated is developed and run on a multi-core computer. The parallel experiments on two multi-core computers with Windows and Linux operating systems, respectively, are fulfilled. The effective integration of the generalized fusion model and a multi-core computer not only yields high speedups but also efficiently leverages the computational resources in a multi-core computer. If the parallel strategies are adopted, in the best cases the fastest times required to finish the entire fusion operation (including disk input/output (I/O) and computation) are close to the time required to directly read and write the images without any computation. The parallel processing implemented on a workstation with two CPUs is able to perform these operations up to13.9times faster than serial execution. An algorithm in the framework is32.6times faster than the corresponding version in the ERDAS IMAGINE software. Additionally, no obvious differences in the fusion effects are observed between the fusion results of different implemented versions.(3) A parallel computing method coupled with the generalized model on GPUA parallel computing method combing the generalized model and GPU is presented for image fusion algorithms. The method in which the setting parameters in CUDA are optimized can process images with large frames and has broad applicability. The experiments with a whole scene of image are fulfilled taking a GPU card with NVIDIA’s Fermi architecture as the computing platform. The results of experiments using whole scenes of images show that the method running on a GPU card currently being middle-performance can achieve speedups up to107×. Moreover, the computing method can be used in many fusion algorithms. Through the analysis of the experimental results, it is found that the factors impacting the speedup are the computation size of a block of data imported into GPU card, and the amounts of exchanged data between host computer and GPU card as well as the performance of the used GPU card.(4) A parallelly optimized block regression based fusion algorithm with the capacity of adjustment of fusion effectsThe parallelly optimized block regression based algorithm (ParaBR) is presented, and the optimal block sizes are selected for optical image fusion and fusion of SAR and optical image, respectively. ParaBR has the capacity of adjustment, which can result in different emphases between preservation of spectral characteristic and enhancement of spatial details according to the users’requirements. It can also achieve a satisfying trade-off between preservation of spectral characteristic and enhancement of spatial details by optimal selection of block size. Compared to five fusion algorithms, ParaBR is stable in fusion effects for optical image fusion. It can also be applied to fusion of SAR and optical image with good fusion effects. The parallelly optimized version with new data fetching strategies only needs the third mount of image reading operations required by the original version, and it can achieve higher speedups and is more scalable than original parallel strategies. ParaBR is the highest one in processing performance among five algorithms which either are commercial version or open source version.The chain including model, algorithm, and computing for image fusion has great advantages in those applications with a large amount data or time limit, like emergent processing of remotely sensed imagery, batch processing for standard products in ground stations, on-boarding image analysis, fast mapping and change detection.