The Research On Multi-Focus Image Fusion Method Based On Deep Learning

With the progress of computer science and the rapid development of sensor technology,the image information on the Internet and life is increasing.However,due to the limitation of depth of field(DOF),that is,the maximum front and rear distance allowed to obtain a clear image,it is difficult to keep all the scenes contained in an image clear,and objects other than focused targets usually fall into defocus state,resulting in clear and blurred areas with obvious boundaries in the image.Among them,the content of fuzzy area is sparse and contains less information,which will seriously affect the human visual perception and the processing quality of computer follow-up work.Multi-focus image fusion is a method to fuse two or a group of images focused on different targets into a fully focused image,which can solve this problem to a certain extent,but different targets often have different scales and detail textures,which are difficult to recognize.Furthermore,the edge blur at the junction of focus and defocus region will lead to the loss of a large amount of information and can not be completely restored.Therefore,the current fusion methods still have great defects.In order to eliminate these effects,make the image more suitable for human visual perception and computational analysis,and improve the efficiency of subsequent digital image processing,this paper has carried out the following research work on multi-focus image fusion:(1)Review,experiment and summarize previous multi-focus image fusion methods,classify them according to traditional mathematical methods and deep learning-based methods,state their advantages and disadvantages,analyze the theoretical reasons for these defects and the improvement direction,and briefly introduce their impact on the work of this paper.(2)At present,there are some problems in the data set of multi-focus image fusion method based on deep learning,such as lack of data,slow training fitting,inaccurate annotation and so on.Therefore,according to different training methods of network model,this paper proposes two new data sets,including foreground focused image,background focused image,target mask and ground truth.Among them,the data set used for regression model training focuses on the fuzzy degree of pixels,the randomness of fuzzy areas and the complexity of the overall scene;The data set used for classification model training focuses on the consistency of scene content,the randomness of target selection and the fineness of mask.They can be used for supervised learning with real image reference,quickly improve the efficiency of network training,minimize the error in the fusion results and make up for the shortcomings of the model.(3)Aiming at the common problems of artifacts,noise,color difference,gray difference,poor texture retention effect,and the neglect of semantic information and gradient information in the existing methods,this paper proposes a fusion network based on regression model and a fusion network based on classification model respectively for the improvement of fusion performance and the further evaluation and exploration of relevant model problems.Among them,the regression model is a small network based on the attention mechanism,and the overall architecture is the pseudo-Siamese network(PSN).The residual atrous convolution pyramid(RACP)is used to extract the multi-scale features of input,share the convolution processing weight of the two images,then input the fusion features into the attention mechanism,and finally use the deep fusion module for fusion,Pixel fitting is used to approach the real clear pixel value to obtain the fused image;The classification model is a large network based on feature pyramid(FP)and semantic segmentation mechanism.It uses dual multiscale FP to extract multi-scale features,which will be combined through residual connection,and then extract the focus / defocus region information through semantic segmentation mechanism to generate segmentation decision map and fusion results.The semantic segmentation mechanism includes an effective channel squeeze excitation(ECSE)module and a channel spatial attention module,which can continuously reduce the dimension of features step by step and calculate the clarity probability of each pixel,so as to realize the pixel binary classification of the whole image.Finally,all fuzzy pixels are covered by a mask to fuse the multi-focus images.In order to verify the performance of all test methods,this paper carried out extensive ablation experiments,involving a variety of data sets,super parameters,network structure and loss function,and conducted a large number of comparative experiments,using quantitative and qualitative evaluation methods: for quantitative analysis,a variety of evaluation metrics without reference and with reference are introduced;For qualitative analysis,a variety of classical traditional methods and advanced methods based on deep learning are introduced.The results show that the fusion model proposed in this paper eliminates various defects as much as possible,and has better performance in both objective metrics comparison and subjective visual comparison.