Research On Multi-Source Image Fusion Method Based On Convolutional Neural Network

Images are an important way for humans to obtain information,and are widely used in military and civilian fields such as battlefield reconnaissance,remote sensing mapping,land management,and medical biology.However,due to differences in imaging methods and imaging conditions,the acquired image data does not fully reflect the complete information of the target.The infrared sensor can obtain target information under dark,smoke and other conditions by detecting thermal radiation imaging,but it is not sensitive to changes in scene brightness,and the visible light sensor collects image data through reflected light,and the obtained visible light image is rich in details and dynamic range.Wide and good visibility,but under the influence of smoke and other interference,the effect of visible light images will be significantly reduced.Therefore,researching multi-source image fusion methods,synthesizing complementary information between images and removing redundant information between images,thereby generating high-quality images,plays an important role in improving the utilization of image information and subsequent image interpretation.However,the multi-source image fusion technology still faces many problems.The feature extraction and fusion rules of the existing multi-source image fusion methods are designed manually.Inaccurate feature extraction methods and fusion rules easily affect the performance of multi-source image fusion.However,the deep learning method can not only adaptively extract low-dimensional features such as the two-dimensional spatial structure of the target,but also fully mine high-dimensional detail feature information in multi-source image data.This paper has carried out research on multi-source image fusion methods based on deep learning in light of the wide application needs of multi-source image fusion.Research on fusion method of multi-focus image and fusion method of optical image and infrared image based on convolutional neural network.The main research contents include:1.Aiming at the problem of the loss of target detail information caused by the manual design of target feature extraction and fusion criteria in the existing multi-focus image fusion method,this paper proposes a multi-focus image fusion based on Full Convolutional Network(FCN)The method uses a deep nonlinear network structure to adaptively extract the fine features of the target and perform precise fusion to generate a high-quality all-focus image.The multi-focus image fusion network based on FCN proposed in this paper is composed of coding layer,fusion layer and decoding layer.First,a fuzzy function is used to process an optical data set to obtain a self-made training and test data set.Then,two source images at different focusing angles are input into the coding layer,effectively extracting high-dimensional fine features of the target in the different focusing images,and fusing the extracted features of the two images through the fusion layer.Finally,the fused features and the features of two source images at different focusing angles are input to the decoding layer for decoding and output.The subjective and objective fusion effect evaluation methods are used to evaluate the fusion quality of different multi-focus image fusion methods on the test set,and the effectiveness and advantages of the proposed method are verified.2.In view of the problem that the existing visible light and infrared image fusion methods require manual extraction of target feature methods,which results in poor generalization performance and adaptive ability,this paper proposes a fusion method of optical image and infrared image based on convolutional neural network.First,a training and test data set for fusion of optical images and infrared images was made by FusionGAN.Then,the visible light image and the infrared image are respectively input into two twin feature extraction networks to adaptively extract the refined features of the target,and the two channel feature extraction results are concatenated according to the feature channel.Finally,the convolutional network is used to fuse features and generate high-quality fusion images.The fusion method of optical images and infrared images based on convolutional neural networks uses Mean-Square Error(MSE)as a loss function for training.Finally,the fused grayscale image is output.Similarly,this paper uses subjective and objective fusion effect evaluation methods to evaluate the fusion results of visible and infrared images obtained by different fusion methods on the test set,verifying the effectiveness and advantages of the proposed method.