Multi-modal Image Fusion Based On Total Variation

Image fusion refers to combining information of source images with same scene into a comprehensive,high quality image.Image fusion is divided into single-modal image fusion and multi-modal image fusion according to the type of sensors of capturing source images.Multimodal image fusion is the most widely used and most difficult type of image fusion.At the same time,this is what this paper mainly research.Multi-modal image fusion,as its name implies,is fusing different source images.Due to the great difference between heterogeneous images,it is difficult to make the fusion image retain the important information of each source image as much as possible while also ensuring that the fusion image has good visual effects,which is also the main task of Multi-modal image fusion.With the continuous advance of technology in information and sensors,multi-modal image fusion has experienced unprecedented development.The types of image fusion algorithms are increasingly rich,and the performance of algorithms has been significantly improved.However,the current multi-modal image fusion algorithm still has many defects.For example,the image fusion algorithm based on multi-scale transformation is inadequate to fuse the source image information.The fusion image generated by the image fusion algorithm based on saliency detection has poor consistency.The detail information of fused image obtained by total variation methods is insufficient.In terms of convolutional neural network fusing methods,it is difficult to train model and the result is unstable.This paper proposes new models and algorithms based on existing image fusion algorithms.And the main work and contributions of this paper are as follows:(1)Through experiments and analysis of the classic multi-scale transform fusing algorithms and the current advanced full-variation image fusion algorithms on the public infrared and visible dataset and the medical dataset,it is found that the variational theory can better integrate Low frequency information of the source images.This paper demonstrates this mindset by designing a series of algorithms and conducting mass comparing experiments.The result shows that the hybrid model algorithms designed in this paper improve the quality of low-frequency information fusion of the multi-scale transformation fusing algorithms and outperform the relative variation methods in merging detail information from source images.(2)Combining the characteristics of saliency detection and variational algorithms,this paper designs a hybrid model image fusion algorithm,the fused images obtained by proposed methods have rich details and good consistency.There are two parts in this method.Firstly,we pre-fuse source images by the particular detection algorithm,and ten adopt the fractional order total variation method what we designed to get the final fusion image by merging pre-fused image and the relative source images.(3)Owing that traditional variation model optimization algorithms have strict requirements on the model,this paper proposes a general total variational optimization framework based on convolutional neural network for image fusion.Under this framework,any convex or non-convex full variation image fusion model can be designed,and the neural network can be used to learn and optimize the solution.This framework can greatly alleviate the limitations of traditional optimization algorithms to the model.The results show that the algorithm proposed in this paper has strong robustness,and the fusion image has obvious advantages compared with the current advanced image fusion algorithm in both objective evaluation indexes and visual effects of the fused image.