Research On Adaptive Image Fusion N Etwork Model Based On Adjacent Feature Fusion

Traditional fusion methods rely on artificial experience to design fusion rules in advance,while deep learning-based fusion methods obtain implicit fusion rules by learning a large number of images.The image fusion model based on deep learning performs better than traditional models in terms of fusion performance.In general,the network models based on deep learning methods are mainly composed of feature extraction,fusion,and reconstruction.However,mainstream deep learning-based fusion methods fail to balance the design of the three stages in the process of network construction,and fail to make full use of the internal information of the training images to improve the fusion effect in the design of loss function.Therefore,this thesis aims to design a stable and robust fusion methods from the two aspects of network construction and loss function design.In the design of network architecture,with the classic yet effective feed-forward denoising convolutional neural network(Dn CNN)backbone network as the foundation,this thesis aim to build a hybrid model of network architecture to balance the three different stages of image fusion.Specifically,the newly proposed dual-branch adjacent feature fusion module adopts the strategy of expanding the number of channels to fully fuse the feature of several adjacent convolutional layers before the middle of the Dn CNN network,enhancing the ability to extract and transmit feature information.In the latter half of the network,the obtained features are reconstructed gradually by reducing the number of channels to get the fusion image.In terms of loss function,in order to make the network better adapt to different scene content images,the gradient feature response values of source images are extracted respectively based on the VGG16 image classification model.After normalization,they are used as the weight coefficients for the source images to participate in the structure similarity(SSIM),mean square error(MSE)and total variation(TV)calculation of three loss function subitems.The proposed model is called adaptive and adjacent feature combination-based fusion network(A~2FNet).To verify the fusion performance of the proposed A~2FNet in different scenarios,it was compared with the current mainstream fusion methods in infrared and visible image fusion(Road Scene,TNO and VOT),multi-focus image fusion(Lytro and MFI-WHU)and multi-exposure image fusion(SCIE and MEFI).Experimental results show that the fusion performance of the proposed model has comprehensive advantages over mainstream fusion methods in widely adopted objective evaluation metrics,and the visual effects are more consistent with human habits.