Research On Algorithms For Pixel-level Infrared And Visible Image Fusion

In recent years,the rapid development of different sensor devices has made it easier for people to obtain various types of image data.However,as the imaging mechanisms of different types of sensors are different,the image information captured by them is also quite different.A single type of infrared image or visible image cannot fully reflect the scene information and cannot meet the actual needs of some applications.For example,the infrared sensor captures the thermal radiation distribution information in the scene better,but there is less texture detail information in the infrared image;on the contrary,the scene information captured by the visible sensor usually includes a large amount of texture detail information,and thus the visible image provides better situational awareness.For some reasons(such as insufficient lighting,occlusion,etc.),the thermal objects hidden in the scene cannot be displayed well in the visible image.Therefore,how to integrate the infrared image and the visible image into one image to facilitate subsequent applications has become an urgent issue.Infrared and visible image fusion(IVIF)technology was born under the above background,and has been widely used in technical fields such as target recognition,target detection,and monitoring.Therefore,it has great practical value in modern military and civilian fields.This thesis concentrates on some research work on some existing problems in the field of IVIF,and proposes two image fusion algorithms based on L2 norm fusion model and a general perceptual fusion framework.The main research contents and contributions of this thesis are listed as follows:(1)In the research of gradient transfer fusion(GTF)algorithms,considering that L1 norm fusion models need to be solved iteratively,a novel fusion algorithm(L2Fusion,L2F)based on L2 norm fusion model is proposed.We convert the fusion task into a minimization optimization problem,and formulate a L2 norm fusion model,where the first term measured by L2 norm tends to constrain the fused image to have the similar intensity distribution as the infrared image,and the second term computed by L2 norm tends to force the fused image to have the similar gradient distribution as the visible image.In addition,we introduce two weights into our objective function to address the smoothing problem of fusion images caused by L2 norm without sparseness.Different from L1 norm fusion model based GTF algorithms,L2 F can obtain the matrix mapping relationship between source images and the fusion result as L2 norm is differentiable,which is effective and efficient.The matrix mapping relationship makes L2 F not only significantly different from the current fusion algorithms,but also lower in computation cost than most fusion algorithms.Experimental results demonstrate that L2 F outperforms L1 norm fusion model based GTF and most state-of-the-art fusion algorithms in terms of visual quality and evaluation metrics,where our fusion results are infrared images with abundant appearance texture information.(2)In the research of perceptual fusion algorithms,considering that the fusion results of traditional multi-scale transform based algorithms are visually unpleasing,a fusion algorithm based on intensity transfer and direct matrix mapping named IT-DMM is proposed.We convert the fusion into a minimization optimization problem and formulate a L2 norm fusion model,and the solution of this model is the direct matrix mapping from source images to the fused image.We use the spatial saliency map of the infrared image to calculate the weights which are important to the performance of our fusion model and are analyzed minutely.Thanks to the efficient weight computation method and the direct matrix mapping,IT-DMM has lower computation cost,and thus it has potential value in practical usage.Qualitative and quantitative experiments demonstrate the advantages of IT-DMM over eleven competitors in obtaining visually pleasing fusion performance,where our fusion results are the visible images with enough infrared thermal object information.(3)In the research of perceptual fusion algorithms,considering that the fusion results of traditional multi-scale transform based algorithms are visually unpleasing,a general perceptual IVIF framework based on linear filter and side window filtering(SWF)technology is proposed.Firstly,we use linear filter and its side window version to decompose source image into edge feature components,hybrid components and base components.Then,the above three components are fused with max-absolute fusion rule and its modified version.Finally,we sum the fused components to construct the fused image.Box filter and Gaussian filter,which are two widely used linear filters,are utilized in our experiments to validate the availability of the presented framework.Experimental results verify the advantages of the presented fusion framework over competitors in obtaining perceptual fusion results.