Research On Multimodal Medical Image Fusion Based On Convolutional Neural Network

Because different modes of medical images have different imaging principles,they present different information.For example,CT can well display bone information,MR image can fully display soft tissue information,PET image can be used for quantitative and dynamic detection of metabolic substances or drugs in human body,providing rich human metabolic information for clinical use.These different images of the mode has its own advantages and limitations,it is difficult to from any single modal about a particular organ or tissue in the image of the complete and accurate information,as a result,combine a variety of modal medical image information,complete the multimodal image fusion,can realize the complementary advantages,multimodal fusion images both retained the features of the original image,and make up for the defects of the single mode state medical image shows more rich details,for the clinical diagnosis and treatment,and provides a strong support for clinical diagnosis and image guided surgery.In this paper,through an in-depth study of the theory of multimodal medical image fusion and the structure of convolutional neural network,the existing problems are analyzed and improved,and the main contents are as follows:In view of existing Residual image fusion method and dense Networks last layer of network only to extract the characteristics of integration,thus lost part of the middle tier to extract the useful information,the problem that fusion image detail and clarity is proposed based on Dual Residual the Hyper-Densely Networks(DRHDNs)multimodal medical image fusion method.DRHDNs include feature extraction and feature fusion.Feature extraction part by combining residual learning and super dense connection constructed a double residual super dense block is used to extract two deep characteristics of source image,residual hop connection ways simplified learning goal and difficulty,achieve integrity,super dense connectivity is extending the densely connected to different paths between layers,thus reducing the middle tier loss of useful information,to complete the preliminary information fusion,residual learn the characteristics and super dense connection is to encourage reuse,the feature extraction is more fully,more abundant details.In the feature fusion part,the two feature images are first stitched together in the channel,and then the fusion image with more details and clearer is finally obtained through reduction and convolution.The network structure of the Dual Residual the Hyper-Densely Networks(DRHDNs)proposed is complex,processing time is long,we puts forward a Gabor filter and Fuzzy logic based CNN multimodal medical image fusion method(Gabor-Fuzzy-CNN,G-F-CNN).G-F-CNN is divided into feature extraction and feature fusion.The feature extraction part is composed of Gabor filter part replacing the convolution kernel of CNN.Gabor filter is used as the fixed weight kernel to extract the inherent features,and has the regular trainable weight kernel.In this way,the two source images are processed and the feature extraction of the source image is completed.In the feature fusion part,the features extracted from the feature extraction part are blurred,inferred and defuzzified,and then the fusion rules based on fuzzy logic are used to complete the feature fusion,and the fusion image with high contrast and clarity is obtained.For the two improved methods,using MATLAB platform and Python related simulation experiment was carried out respectively,the experimental results are compared with the related fusion methods in recent literatures,from the objective indicators and subjective visual effect of the fusion is analyzed,the method has good performance and demonstrate the effectiveness of the proposed method in this paper.