Research On Infrared And Visible Image Fusion Methods Based On Multiscale And Saliency Region Analysis

Image fusion aims to combine multiple input images from different modal sensors into a single image to enhance the understanding of the scenario.Infrared and visible image fusion is an important fold of the image fusion.It makes full use of the complementarity of infrared and visible image information for fusion.The fusion result not only provides more comprehensive information about the related scenes,but also conduces to the subsequent applications such as target detection and target tracking.Fusion algorithms based on multiscale analysis and saliency region analysis are two important types of algorithms in the field of infrared and visible image fusion,because they can well represent the inherent characteristics of the image and help improve the visual effects of the final fusion image.However,these two types of algorithms also have some problems in the actual infrared and visible image fusion process.Many scholars have also proposed relevant improved algorithms to solve these problems,but currently there is no algorithm that can solve all the problems that may be encountered during the fusion process.One method is to improve one aspect of the problem generally.In this paper,we design four fusion methods for four typical and common problems encountered in the actual engineering application of these two types of algorithms and the main research contents are as follows:1.Aiming at the problem that conventional multi-scale analysis methods rely on predefined transformation methods and lack stability and flexibility when facing with uncertain scenes in the source image,we proposed a two-objective function optimization criterion based on the improved fusion global-local-topology partical swarm optimization(FGLT-PSO)to design an adaptive weight fusion rule,so that the algorithm can unsupervisedly determine the final fusion image of the two source images without prior knowledge.Meanwhile,in order to measure the stability of the algorithm,Rank Score:a scoring mechanism based on statistics was proposed.Compared with other algorithms,the Rank Score indexes of our method are the best in the experiments,indicating it can ensure stability and good fusion effect when processing different types of images.At the same time,the latent low-rank representation(LatLRR)is applied in a multi-scale framework,while fully retaining the advantages of the multi-scale analysis algorithm,it uses low-rank constraints to effectively remove a certain degree of noise.The proposed algorithm significantly outperforms other comparison algorithms on the N~AB/FB/F index,proving that the algorithm can overcome the noise sensitivity problem of general multi-scale analysis fusion algorithms.2.Aiming at the problem that common multi-scale fusion algorithms can not keep the contour information and the texture detail information in the contour at the same time,which leads to the loss of detail information or the contour blur,multi-scale image fusion algorithm based on the improved co-occurrence filter is proposed.Firstly,we design a multi-scale and multi-directional transformation method based on the improved co-occurrence filter to decompose the source images.This decomposition method combining co-occurrence filters with discrete compactly supported shearlet transform decomposes the contour information and the texture information into different sub-images and well represents the large scale outline information in the base layer,which is conductive to the subsequent specific processing of two kinds of detailed information.Then,according to the characteristics of the two sub-images,specific fusion rules are designed for each sub-image fusion.Experimental results show that the method can maintain the contours of infrared and visible images and enhance detail representation at the same time.3.In the view of the fact that it is difficult to simultaneously and effectively extract saliency target regions of different scales in the source image with a complex background,an image fusion algorithm based on multi-feature salient region analysis is proposed.Firstly,aiming at detecting as many more complete saliency target areas as possible,we use a variety of feature information to compose the input value of the phase spectrum of quaternion fourier transform model and use the gabor filter to generate the best saliency map.Then an improved GrabCut algorithm is proposed to segment the image into salient and non-saliency parts,and different fusion strategies are used for the two parts in the subsequent fusion process.Experimental results show that the algorithm can effectively handle the scenes with complex background and multi-scale targets.4.It is difficult to compromise the speed and the fusion quality in the normal fusion method based on saliency region analysis.To this end,we proposed an image fusion algorithm based on infrared salient region extraction and non-subsampled shearlet transform(NSST).According to the characteristic of the infrared image,we design global and local saliency map to detect the salient target area of the infrared image and map the detection result to the visible image at the same time,which helps to highlight the infrared target in the result and can improve the segmentation efficiency.Meanwhile we design the fusion strategies based on NSST,which can compensate the loss of saliency information in the visible light image caused by the segmentation and improve the visual performance of the fusion result.The proposed method combines multiscale analysis-based and saliency region analysis-based fusion algorithms,which can balance the algorithm efficiency and fusion quality.Experiments show that the algorithm is easy to operate and the fusion quality is good.In addition,the speed of the method can meet the real-time engineering requirements.5.At the end of the paper,Rank Score is used to comprehensively analyze and compare the fusion performance of the four algorithms proposed in this paper and eight comparison algorithms.According to the performance,the applicable scenarios of the four algorithms proposed in the paper are indicated.The paper focus on the problems faced in the theoretical research and practical application of infrared and visible image fusion methods based on the multiscale analysis and salient region analysis,we proposed four new image fusion algorithms,which not only enriches the theoretical system of infrared and visible image fusion but supplies the foundation for practical applications.