Research On Multi-focus Image Fusion Algorithm Based On Deep Learning

As a branch of information fusion,image fusion is one of the hot topics in current fusion research.In dealing with the problem of multi-focus image fusion,how to extract more features from the two images and fuse them to obtain a more accurate decision map is the key to solving this problem.For decades,many researchers have proposed a large number of image fusion algorithms.In the process of taking digital photography,different apertures and focal lengths makes it difficult for many digital devices to take an image in which all the objects in the scene at different depth-of-field are focused.Usually,only the objects in the depth-of-field are clear,and other objects are more likely to be blurred.To solve this problem,multi-focus image fusion technique has been emerged.Although the information in any single image cannot explain the whole scene,the information contained in multiple images of the same scene is complementary,so multi-focus image fusion can be synthesized by extracting features from different focused images and designing corresponding fusion rules to generate a clear image which contains as much information as possible.In the past few years,many multi-focus image fusion methods have been proposed and these methods can be roughly classified into two types: spatial domain and transform domain.Besides,in recent years,deep learning has achieved great progress in fields of computer vision and image processing problems such as target detection and image segmentation,etc.The application of deep learning methods for multi-focus image fusion has also emerged as an active research.However,traditional image fusion methods require focus detection on pixels in the image.This process often uses block and region-based methods,so the quality of the fusion is affected by the division method,and the problem of unsatisfactory edges often occurs.In addition,more complex fusion rules need to be designed to fuse images.The subsequent imagefusion methods based on deep learning often have a single network structure,and can not achieve end-to-end pixel-level prediction during the training process.The fusion results often appear artifacts at small edges.This paper focuses on the limitations of traditional methods for focus detection and fusion rules,and the problem of single network structure and label design in deep learning.The main research contents are as follows:(1)First,this paper proposes a CNN architecture for multi-focus image fusion,Pyramid Pooling Dense Convolutional Neural Network for Multi-focus Image Fusion.We can get a model that obtain the initial decision map by training the network,then we get our final decision map after post-processing based on initial decision map,finally we use the source image and the final decision map to obtain a fused image.In order to make full use of the global information from small regions in the source image,we adopted the pyramid pooling module to extract features from different small regions.In addition,1×1 convolution is utilized after each layer to reduce the dimension of context features.In order to make full use of the information from each previous layer in the network,a dense connection block is designed,and the information of each layer in this block is not only transmitted to the next layer but also to the subsequent layers.(2)In addition,in order to further optimize the network structure and obtain a more accurate decision map,in addition to extracting feature information from the source images,this paper uses the generative adversarial network structure,the source images are concatenated as inputs of the generator,and then we distinguish the output fake samples of the generator and the real samples by using the discriminator,finally we train our generator to get a result which close to the focus ground truth.In addition,in this paper,we improved our loss function in the training process of the generative adversarial network structure,and the content loss of the image is added to solve the problem that the traditional generative adversarial network loss function affects the quality of the generated image.The results show that compared with the recent existing algorithms,the proposed multi-focus image fusion algorithm has better edge effects and performs better on objective quality evaluation metrics.