Medical Image Fusion Algorithm Based On Convolutional Neural Network

In recent years,many medical imaging technologies have been widely used to display various information in different ways to identify diverse diseases in the human body.Medical images of different modalities can display complementary information of the human body.The main purpose of medical image fusion is to fuse two or more medical images of different modalities into one medical image containing comprehensive information.Medical image fusion plays an indispensable role in clinical diagnosis and disease treatment.Doctors can utilize the fused medical images to make disease judgments and perform clinical treatment more comprehensively and accurately.In addition,image fusion needs to ensure that all important information in any input should be preserved,the fused image should not produce artifacts,and the fused image should have stronger robustness.Multi-modal medical image fusion mainly includes three levels: pixel-level fusion,feature-level fusion and decision-level fusion.Pixel-level fusion can retain more information of the source images and is easy to implement.Therefore,this paper focuses on pixel-level fusion.Pixel-level image fusion can be divided into transform domain fusion methods and spatial domain fusion methods.Traditional transform domain methods often have problems such as image distortion and loss of structural information.The spatial domain algorithms are simple in calculation,low in operation complexity,and can retain more information of source images.However,they often produce more serious blocking effects.With the development of deep learning,convolutional neural network(CNN)is widely used in image fusion at present.In order to solve the problems of the above methods,this paper proposes two medical image fusion algorithms based on convolutional neural network,which have achieved excellent image fusion results.The main contents of this paper are as follows:(1)Multi-modal medical image fusion based on rolling guidance filter with CNN and nuclear norm minimizationIn order to suppress the blocking effects and artifacts in fusion results,a multi-modal medical image fusion algorithm based on CNN feature mapping and nuclear norm minimization(NNM)combined with rolling guidance filter(RGF)is proposed.First,RGF is used to decompose the medical images into base layer components and detail layer components.Secondly,the pre-trained CNN model is adopted to fuse the base layer components to obtain the base layer fused image.Then,the detail layer components are processed by NNM to get the detail layer fused image.Finally,the fused base and detail layer images are merged into the fusion result.Experimental results show that the proposed algorithm has achieved better results in visual evaluation and objective evaluation compared with the state-of-the-art medical image fusion algorithms.(2)Two-scale multi-modal medical image fusion based on structure preservationDue to the advantages of structure preserving filter and deep learning in image processing,a two-scale multi-modal medical image fusion algorithm based on structure preservation is proposed.The algorithm first adopts two-scale decomposition to decompose the source images into base and detail layer components.Secondly,iterative joint bilateral filter(IJBF)is used to fuse the base layer components.Thirdly,the local similarity of the images and CNN are used to fuse the detail layer components.Finally,the two-scale image reconstruction is adopted to obtain the final fused image.The experimental results show that the proposed algorithm has better image fusion effects than the most advanced medical image fusion algorithms.