Study On The Theory Of Multi-Sensor Image Fusion And Its Applications

Multisensor image fusion is a kind of information fusion, which refers to the synergistic combination of different sources of sensory image into a single image. The information to be fused may come from different sensors of the same object or scene,or from the same sensor of different imaging manner, or from different images of different time period. The fused image can reflect multiple properties of source images, which makes it more suitable for the purpose of human visual perception and computer processing tasks such as medical applications, detection or classification tasks.Multisensor image fusion is a new branch of research that involves sensors, signal processing, image processing, and artificial intelligence. Recently, image fusion has been an important and useful technique for image analysis and computer vision.Multisensor image fusion can occur at three different levels:pixel level, feature level, and decision level. However, the main concern of this thsis is to present a study on pixel level fusion, feature level fusion and the evaluation method of image fusion respectively. The main contributions are summarized as follows:(1) Three new algorithms are proposed in pixel level fusion. The first new algorithm of pixel level fusion is proposed based on the combination of windows and vector evaluated quantum behaved particle swarm optimization(VEQPSO), in which VEQPSO incorprating with gray relational analysis is utilized for low frequency band, while information energy and fuzzy entropy of windows are adopted for fusion of high frequency band. Since there is position error which leads to the downward overflow of gray value of images when the scale signal is reconstructed for modified morphorlogical wavelet filter (MMWF), a detection-refusion strategy is proposed in the second proposed algorithm. The proposed method preserves the advantages of MMWF including fast speed, effectiveness, and easy implementation. Furthermore, the performance of the proposed fusion method is improved significantly. To reduce the dependence on parameter and strengthen the robustness of the algorithm based on statistical model, a modified statistical fusion model (MSFM) is proposed in which new objective function is presented and the constraints are simplified. Using this new model, the interrelated spatial information is well enhanced, and the spectral information of multi-spectral images is effectively kept. Furthermore, the proposed algorithm can not only avoid determining a threshod of the conventional statistical model, but the robustness is enhanced and the complexity is reduced as well.(2) Four algorithms are proposed in feature level fusion. A noisy image fusion algorithm based on Kalman filter is proposed based on previous image fusion method using multi-feature fuzzy clustering. The proposed algorithm gains an improved performance since it combines the advantages of Kalman filter and multi-feature. A new region similarity is proposed based on multichannel Gabor filtering and region fusion method. The research indicates that the performance of image fusion is insensitive to the selection of different Gabor parameters including the center frequency and orientation. Furthermore, the proposed method for image fusion is stable in fusion performance. To avoid the local minimum problems of fuzzy C-means(FCM), a new clustering algorithm QPSO-FCM is proposed that incorporates the FCM into QPSO algorithm. Since QPSO-FCM can produce a very good segmentation, which leads to an improved performance of fusion. A new double fusion method based on multi-feature is proposed. The research indicates that the proposed method is very effective for the fusion of multi-focus images.(3) After research on the conventional evaluation methods of image fusion, a novel metrics for evaluation of fused images are proposed based on the similarity of corresponding regions in images. The new metrics are computed on a region-by-region basis, which is more suitable for the evaluation because human eyes are more sensitive to regions. The region information is represented by feature matrix of region, which consists of multi-feature vectors including spatial information, texture and gray value, which can adequately reflect the regional content. Two new quality metrics are proposed which consider the reference image is available or not. Furthermore, two dimensional principal component analysis(2DPCA) is introduced into the process of evaluation, which leads to a new version of the proposed evaluation method. Research indicates that the proposed metrics are more consistent with the nature of human perception as it considers the local image variations and the saliency of region.