Moulti-focus Image Fusion Algorithm Based On Convolutional Neural Networks

Due to the limitation of the depth-of-field of the optical system,the visible light imaging system has limited focus on the target area,and only the part neighbouring the focus part of the generated images in the same scene is clear.Multi-focus image fusion refers to the image fusion processing of two or more differently focused images,making the fused image more beneficial to the human recognition and computer processing and analysis,which has been widely used in target recognition,machine vision,digital cameras and other fields,so the realization of multi-focus image fusion has very practical significance.Currently,there are mainly two types of multi-focus image fusion methods.One is spatial-based image fusion method,such algorithms generally perform faster,but usually there are block effects in the fused image.The other is based on transform domain,the fused images are often accompanied by image warping,artificial texture and other phenomenon.This paper aims at the existing problems of multi-focus image fusion algorithms,using convolutional neural networks to avoid the complex pre-processing of the image,and extract the features of the original images,proposing two multi-focus image fusion algorithms based on convolutional neural networks.In addition,in order to better evaluate the quality of images obtained by image fusion algorithms,this paper also proposes an objective quality assessment algorithm of image fusion based on bispectrum analysis.The main research is as follows:(1)Multi-focus image fusion algorithm based on non-subsampled shearlet transform domain via convolutional neural networksIn order to solve the problem of distortion and information loss in multi-focus image fusion,this paper proposes a multi-focus image fusion algorithm based on NSST via convolutional neural networks,combined with the merits of time-frequency separation of NSST.Firstly,the source image is NSST decomposed.Secondly,the decomposed low-frequency coefficients are fused based on convolutional neural networks,and then fusing the decomposed high-frequency coefficients based on guided filtering via large model ofSum-Modified-Laplacian.Finally,the fused coefficients are transformed by NSST inverse transform to get the fused image.The algorithm not only fully preserves the source image information but also improves the image space continuity.(2)Multi-focus image fusion algorithm based on unsupervised convolutional neural networksSince CNN performs supervised training,in the multi-focus image fusion,we do not need to know the correspondence between the original images and the result images.Our goal is only to get the characteristics of the source images with the distinction between clarity.Therefore,this paper proposes a multi-focus image fusion algorithms based on unsupervised convolutional neural networks.The simple two-layer convolutional neural network is used to extract the features of the source images and the fused image is obtained by the fusion rule of the larger 2-norm of the sparse vectors.The experimental results show that our algorithm is superior to other algorithms in both subjective visual and objective evaluation.(3)An objective quality assessment algorithm of image fusion based on bispectrum analysisAiming at the existing image objective criteria ignoring the phase information of the image,this paper proposes an objective quality assessment algorithm of image fusion based on bispectrum analysis on the basis of the analysis of the signal bispectrum characteristics.The objective criteria proposed in this paper can effectively evaluate the quality of image fusion and accord with human visual perception.The experimental results show that our objective criteria overcomes the problem that other objective criteria cannot effectively evaluate the quality of fused images under extreme circumstances.Finally,this paper also uses the MATLAB GUI to generate an image fusion demonstration software for the above algorithm,which can more simply and intuitively demonstrate the effects of the fusion algorithm and the objective criteria.