Research On Multi-focus Image Fusion Based On Convolutional Neural Network

In the field of digital image processing,digital imaging systems have limited control over the depth of field because of the limitations of digital image sensors.Under the same scene,the captured images will show clear areas and blurred areas at the same time,which named multi-focus images.The task of extracting and fusing the clear areas of a group of multi-focus images into a full-resolution image is called multi-focus image fusion.How to minimize the loss of fusion details is the research hotspot and difficulty in the field of multi-focus image fusion.Classical fusion methods,such as the spatial domain methods or the transform domain methods,have simple fusion strategies,but their fusion results are prone to block effect or artifacts.Emerging fusion methods,such as the fusion method based on sparse representation,have good performance but their computational complexity is high,and they may produce the loss of boundary information in the fusion results.The main research purpose of this paper is to resolve the above fusion problems by introducing the deep learning algorithms into the classic fusion methods.Aiming at the problem of the loss of boundary details in the fusion method based on convolutional neural network,this paper proposed a multi-focus image fusion method based on multi-convolutional neural network in the non-subsampling contourlet transform(NSCT)domain.The main research contents of this paper are as follows:(1)Aiming at the basic concept of multi-focus image fusion,this paper introduces the traditional multi-focus image fusion algorithms and emerging fusion algorithms.Firstly,we introduce the basic principles of multi-focus image fusion.Secondly,the core ideas of traditional multi-focus image fusion methods and emerging fusion methods are elaborated.Then,the advantages and disadvantages of the representative fusion algorithms are analyzed and summarized.Finally,the subjective and objective evaluation indexes are introduced.(2)The preprocessing process of the data are descripted in detail,including dataset production and dataset preprocessing.Firstly,the image fusion process is introduced,which explains the importance of data preprocessing in the fusion process.Secondly,three dataset production methods are introduced in detail,and the applicable scenarios are analyzed.Finally,two data preprocessing methods are introduced,and the application scenarios of the two preprocessing methods are obtained through experiments.(3)Aiming at the problems of the existing multi-focus image fusion algorithm,a multi-focus image fusion method based on multiple convolution neural networks in the non-subsampling contourlet transform domain is proposed.Firstly,the NSCT algorithm is used to decompose the source images to obtain a group of multi-level and multi-directional sub-band decomposition images,which aims to retain more detailed information.Secondly,the focus score map for each level is obtained by classifying the focus and defocus areas of each detail image with the CNN and SoftMax loss function.Then,some post-processing operations,such as binarization,consistency verification and boundary blurring,are performed on the classified focus map to obtain the final decision map.Next,the fused detail images for each level are obtained by using the corresponding final decision map.Finally,the NSCT reconstruction method is used to obtain the fusion image.The proposed algorithm and comparison algorithms are tested on a series of datasets.From the subjective evaluation and objective evaluation indicators,the proposed algorithm shows better fusion performance than other algorithms.