Research On Image Fusion Method Driven By Synthetic Data

Image fusion is an image enhancement technology that combines images obtained by different types of sensors to generate fused images with rich information and good robustness for subsequent image processing.Image fusion technology is widely used in the fields of military,remote sensing,security surveillance and medical imaging.The key to the design of the fusion method is efficient image information extraction and appropriate fusion rules,and to avoid the influence of artificial factors on the fusion result.Traditional image fusion algorithms are based on hand-designed fusion rules,which are complex and slow,with poor generalization ability and robustness.With the application of deep learning in image fusion,manual design of fusion rules is avoided.GAN(Generative Adversarial Networks)is a typical application of deep learning in image fusion.It uses generators to extract infrared and visible image features for fusion,and antagonizes the discriminator to optimize generator training.Due to the lack of labeled data,almost all previous image fusion networks are trained in an unsupervised manner,and some unsupervised methods are difficult to balance the two modalities of infrared and visible images,so a batch of infrared and visible images are synthesized for a supervised network Training is necessary.The main research contents of this paper are as follows:(1)In the past fusion networks,unsupervised training is the mainstream method of image fusion.Due to the lack of labeled data sets,there is no GT(Ground Truth)to guide fusion.In order to overcome the difficulty of the lack of tags,this topic synthesizes a set of infrared and visible data sets with GT to guide the training of the network.This research uses the RGBD dataset NYU-Depth from New York University as the basic data of the synthetic image.Through the optical transmission model,according to the infrared thermal radiation imaging mechanism,the infrared image has strong anti-interference ability against weather,and the depth map simulates the transmission of light.The loss to simulate infrared image and visible image.(2)Since there is no real fusion image as a reference,the existing image fusion method lacks the fusion image as a supervision condition,and the training method based on supervised learning is difficult to apply to image fusion.The existing fusion network is as far as possible to find between the two modes balance.Based on this,an infrared and visible image fusion method based on the optical transmission model is proposed.The edge and detail loss function is designed in the conditional GAN based on the synthesized data set,and the synthesized multimodal image data set is used to train the image in an end-to-end manner.Network,and finally get a converged network.The network can make the fusion image better retain the details of the visible image and the target characteristics of the infrared image,and sharpen the boundary of the thermal radiation target in the infrared image.Compared with mainstream IFCNN,Dense Fuse,Fusion GAN and other methods on the TNO public data set,the effectiveness of this method is tested through subjective and objective image quality evaluation.The proposed method has achieved good fusion effects in the fusion of infrared and visible image and multifocus image.(3)Due to the different characteristics of infrared and visible imaging,there are more outline structure information in the infrared image,and more texture detail information in the visible image.In the previous fusion framework,infrared and visible images are stacked into the nerves.In the network training,general features are extracted,ignoring the respective features of infrared and visible image.Therefore,under the supervised fusion framework,the DCGAN(Deep Convolutional Generative Adversarial Networks)is used to obtain the infrared image structure information and the visible image texture information respectively,and then exchange the infrared image and the visible image texture information for fusion.On this basis,the result of the generator and the discriminator are gamed,and the fusion image is finally generated.Compared with the mainstream fusion method on the TNO public data set,the effectiveness of the method is tested through subjective and objective image quality evaluation.