Fusion Technology Of Infrared And Visible Images

Infrared and visible imaging sensors are widely applied in military, security and surveillance fields, hence the majority of image fusion research, which has been conducted by USA and European countries, has focused on infrared and visible images. The research presented in this thesis is concerned with the fusion technology of infrared and visible images. The aim is to develop computationally efficient, high-quality image fusion methods, and to provide some new fusion approaches of infrared and visible images, which are suitable for practical applications. According to the color of the outputs, the fusion schemes of infrared and visible images can be classified into two groups: grayscale image fusion methods, and color image fusion methods. Both of them are considered in this thesis, which mainly consists of three parts and respectively investigates three fusion approaches: (grayscale) weighted average multiresolution image fusion methods, fusion methods of infrared and color visible images, and color transfer based image fusion methods.According to the fusion rules, multiresolution image fusion schemes can be divided into maximum multiresolution image fusion methods, and weighted average multiresolution image fusion methods. Compared to the former approaches, the later methods exhibit better fusion performance because weighted average multiresolution image fusion methods provide more accurate processing when the features of the source images are similar. However, the selection and calculation of the weights in the later methods are more complex than those in the former approaches, and consequently, the computational complexity of the later methods is much higher. The objective of the first part in this thesis is to simplify the algorithm architecture of weighted average multiresolution image fusion methods, and to develop more computationally efficient algorithms with the premise that the good fusion quality is preserved. Chapter 2 and 3 are devoted to this objective.Chapter 2 first analyzes Piella's multiresolution image fusion framework. Following this analysis, Chapter 3 extends the model of weighted average multiresolution image fusion within Piella's framework, and presents the idea of controlling the decision map only with the match measure, thus simplifys the algorithm architecture of weighted average multiresolution image fusion methods. Depending on the new model of weighted average multiresolution image fusion, with the correlated signal intensity ratio as the match measure, the chapter presents four different weighted average multiresolution image fusion methods: T-RWAM, L-RWAM, B-RWAM, and E-RWAM. Compared to conventional weighted average multiresolution image fusion approaches, the proposed methods exhibit better fusion performance, and can achieve much lower computational complexity.The grayscale image fusion techniques are usually required for merging infrared and color visible images. In order to obtain high-quality fused imagery, multiresolution image fusion methods are generally employed to implement grayscale fusion. This leads to a large amount of complex computations, and thus presents a significant challenge when real time implementation is considered. The objective of the second part in this thesis is to develop a fusion method of infrared and color visible images, such that using the simple pixel averaging scheme to perform grayscale fusion can also provide a good fused result. Chapter 4 is devoted to this objective.Chapter 4 presents a luminance-contrast transfer approach, which is easily implemented, fast to execute and has the advantage of rapidly improving the luminance and contrast of grayscale fused imagery. Based on this technique, the chapter proposes a fusion method of infrared and color visible images, named LCT. This approach can be carried out in two ways: the standard LCT method and the fast LCT method. The standard LCT method requires a linear luminance- chrominance color transformation for performing the fusion process. The fast LCT method is the simplification of the standard method, and avoids color space conversion, instead, it manipulates images directly in RGB space. The LCT method can produce a bright, high-contrast color fused image with similar natural color characteristics as the input color visible image, and even the P-LCT approach, which employs the pixel averaging scheme to perform grayscale fusion, can also yield a good color fused result.At the core of all the existing color transfer based image fusion methods is the statistical color transfer technique proposed by Reinhard et al. This technique is able to effectively recolor color fused imagery, and has the advantage of giving color fused imagery a natural daytime color appearance and maintaining good color constancy. However, the lαβtransformation used in Reinhard's color transfer method exhibits high computational complexity, in addition, lαβspace does not facilitate image fusion operations, either. One important objective of the third part in this thesis is to simplify lαβinto a linear color transformation particularly suitable for image fusion, in order to develop an efficient and effective color transfer technique. Chapter 5 is devoted to this objective.Chapter 5 first presents and discusses the properties of Reinhard's color transfer technique, and then extends the Y(B-Y)(R-Y)-type color space into color transfer. Depending on YCBCR space and Reinhard's statistical color transfer strategy, the chapter proposes the YCBCR color transfer method. This approach not only has a low computational complexity, but it is also very suitable for color image fusion. That facilitates easy enhancing the luminance contrast of the final color fused image by replacing the luminance component of the color fused image with the grayscale fused image. The proposed method is able to give color fused imagery a natural daytime color appearance, and has better capability of recoloring color fused imagery than Reinhard's approach. The proposed method can also be successfully applied to recoloring a variety of color images.Extending the color transfer technique directly into image fusion is not the most efficient way for processing. The procedure of first constructing the source RGB color fused image and then recoloring it requires forward and backward color space conversions. Even though color transfer is implemented in a linear color space, the color space transformation also needs numerous multiplication and addition operations. The other important objective of the third part in this thesis is to reduce the operations of coordinate conversions and develop a fast color transfer based image fusion method. Chapter 6 is devoted to this objective.Based on the work of Chapter 5, Chapter 6 presents a fast color transfer based image fusion method, named AOCTIF. This approach skips the transformation from RGB to YCBCR space, and directly uses the grayscale fused image and the simple difference signals of the input images to form the source YCBCR components. The chapter develops two solutions, namely the P-AOCTIF and M-AOCTIF methods, to fulfill different user needs. The P-AOCTIF method, which employs the pixel averaging fusion scheme to perform grayscale fusion, answers to a need of easy implementation and speed of use. And the M-AOCTIF method, which employs the multiresolution fusion scheme to perform grayscale fusion, answers to the high quality need of the fused products. The AOCTIF method offers an effective way to construct a fused image with a natural daytime color appearance. Even using the simple P-AOCTIF method can also produce a satisfactory color fused result.Increasing interest is being shown toward practical image fusion technology. This thesis offers some fusion techniques of infrared and visible images, which can be used in practical applications. Some proposed methods are not only able to produce high-quality fused imagery, but also easy to implement and time efficient. The work in this thesis is also helpful to image fusion in other fields such as remote sensing and medical imaging.