Multi-focus Image Fusion Based On NSST And Convolutional Neural Network

When people use a camera to obtain pictures,due to the limitation of the camera's depth of field,it is often impossible to obtain a fully focused image in the same scene.The usual method is to focus and shoot images at different positions,and then extract the respective clear areas of the two or more images and combine them into a fully focused and fully clear image.This is multi-focus image fusion.After years of development,multi-focus image fusion has formed four types of methods based on transform domain,spatial domain,combination of transform domain and spatial domain,and deep learning.Each type of method has its own advantages and disadvantages.Aiming at the shortcomings of existing methods,this paper proposes a fusion method based on multi-scale convolution features and a method combining NSST and convolutional neural network.The main contents of this paper are as follows:(1)Aiming at the existing convolutional neural network fusion method that only uses the last layer of feature maps,while avoiding the complicated label production process,this paper proposes a multi-focus image fusion method using different scale feature maps.First,the network structure of encoding and decoding is used for network training,and the SKNet attention module is embedded in the encoding network to enhance the feature extraction ability of the network.Then in the fusion stage,the three-scale features in the coding network are respectively calculated for focusing attributes,voting to determine whether to focus or not,an initial decision diagram is generated,and the initial decision diagram is optimized to obtain the final decision diagram.Finally,the fusion image is generated according to the final decision diagram.Experiments show that this method can accurately extract the focus area and obtain a high-quality fusion image.(2)In view of the insufficient ability of convolutional neural network to retain edge details,NSST can decompose images into multiple scales,which is in line with the characteristics of the human visual system.It is the most advanced multi-scale transformation method at present,so this paper proposes a NSST and convolutional neural network.Multi-focus image fusion method combined with network.First,the source image is decomposed by NSST,then the low frequency subband is fused by the trained low frequency network,and the high frequency subband is fused by the high frequency network,and finally the inverse transformation of NSST is performed to obtain the final fused image.After comparison experiments of multiple sets of images,this method has achieved better fusion effects than comparison algorithms.