| Multisensor image fusion has become an area of intense research in the past few years. A variety of image fusion approaches have been studied and applied to varied applications. However, there have been very few attempts to employ rigorous estimation theory in image fusion. In this thesis, we propose a statistical signal processing approach to image fusion based on the Expectation-Maximization (EM) algorithm. This method utilizes a generalized distortion model, the Gaussian mixture model, for fusion. This EM-based fusion algorithm is also extended to fusion of video sequences by considering optimum use of neighboring frames. The video fusion method is demonstrated to have significant advantage in terms of sensor noise reduction. The estimation theory based methods are able to include prior information to obtain a more reliable fused result. We develop a novel image fusion method based on the hidden Markov model (HMM), which exploits the correlations between the wavelet coefficients across different scales. In this method, the fusion is performed over wavelet trees spanning the subbands of all wavelet scales, while in most other wavelet-based fusion methods, the fusion is performed subband by subband. Besides these pixel-level fusion methods, we also describe a new region-based image fusion method using the EM algorithm. This method takes advantage of the similar intensity or texture in a region for fusion. Experiments show the advantage of this method in dealing with region interface artifacts. We demonstrate the efficiency of these estimation theory based fusion approaches by applying them to fusion of visual and non-visual images with emphasis on concealed weapon detection and night vision applications. Registration must precede our fusion algorithms. We propose a robust non-rigid image registration algorithm that uses geometric features and salient region features. This algorithm is a hybrid approach co-optimizing point-based and image-based terms, and is built to be extremely robust to feature extraction errors. The experimental results demonstrate the robustness and accuracy of the algorithm and its capability of handling a considerable number of outliers. |
