| Recent years have witnessed that the multimodal medical image fusion(MMIF)plays critical roles in the research of human brain structure,lesion location,illness diagnosis,treatment planning,etc.Medical images are often with different modalities,which can be classified into anatomical and functional.Anatomical imaging modalities include computed tomography imaging,magnetic resonance imaging,and ultrasonography imaging,which provide morphologic details of human body.Functional imaging modalities such as single photon emission computed tomography and positron emission computed tomography provide metabolic information without anatomical context.Single modality medical images have limitations in the description of local detail information and are insufficient to provide physicians with adequate information for disease diagnosis.So,multimodal medical image fusion offers an important approach by integrating complimentary features of different imaging modalities into one fused image.In clinic,it still requires professional doctors,according to their accumulated experiences,to choose the appropriated fused images from numerous candidates by visual inspection,which is lack of benchmark evaluation criteria for medical image fusion metrics.Existing objective quality assessment methods are not designed for medical fusion images.They have great limitations in evaluating multimodal medical fusion images.Furthermore,it is difficult to be incorporated into the design and optimization of image fusion.Therefore,multimodality medical image fusion and quality evaluation have important clinical value.The research work mainly focuses on the key technical problems of multimodal medical image fusion and quality evaluation.Based on the in-depth study of the fusion problem,a transform domain fusion algorithm and a space domain fusion algorithm are proposed.Existing objective quality assessment methods have great limitations in evaluating multimodal medical image fusion.Quality assessment metric for medical fused images is proposed.The evaluation results are employed for the performance assessment of fusion algorithms,dynamic optimization and adjustment of parameters,towards improving the performance of image fusion.The main research content and innovation results of this dissertation are as follows:(1)Perceptual multimodal medical image fusion based on internal generative mechanism(IGM)is proposed in this dissertation.Researchers have found that human visual system(HVS)is a complicated system.Vision generated by HVS is not a straight translation of the ocular input,but a result of active inference of brains,such as the bayesian brain theory indicate that the brain works with an internal generative mechanism for visual information perception and understanding.Inspired by IGM,a perceptual image fusion method is proposed that employs IGM.Firstly,source images are divided into a predicted layer and a detail layer by simulating the mechanism of human visual system perceiving images.The prediction layer represents the approximate component of the source image,and the averaging strategy is applied to the fusion layer.At the same time,the detail layer reflects the texture or edge information of the image.The tchebichef moment is very effective in the shape representation and can effectively capture edge features,the energy of moments for blocks are used to fuse the detail layer.The fused image is finally obtained by combining coefficients in all the fused layers.In the experiment,five objective fusion evaluation strategies were used to evaluate the fusion image.Experimental results prove that the proposed fusion algorithm is superior to the previously developed methods.(2)Multimodal medical image fusion based on discrete tchebichef moments energy motivated adaptive pulse coupled neural network(PCNN)is proposed in this dissertation.In many image fusion methods based on PCNN,normalized coefficients are used to motivate the PCNN,and this makes the fused image blur,detail loss and decreases contrast.Moreover,they are limited in dealing with medical images with different modalities.Although the modified spatial frequency be used as the input in several medical image fusion algorithms with encouraging results.However,they are more sensitive to directional than the edge information.The aforementioned problems were solved by using edge features to motivate the PCNN since human eyes are more sensitive to the edge information,furthermore medical images of different modalities contain large amount of edges features,which are quite often very subtle in nature.Discrete tchebichef moment is very efficient in shape representation,it can effective capture edge features.Firstly,medical images are divided into equal-size blocks and the tchebichef moments are calculated to characterize image shape,and energy of blocks is computed as the sum of squared non-direct current moment values.Then,energy of tchebichef moments for blocks(ETMB)is used to motivate the PCNN.In order to adapt to different modalities medical image fusion,adaptive linking strength and adaptive threshold of PCNN are proposed.Finally,large firing times are selected as coefficients of the fused image.The proposed method is advantageous over the existing methods for processing medical images with different modalities.(3)Objective quality assessment for multimodal medical image fusion is proposed in this dissertation.As well known,current image fusion quality metrics are not designed for MMIF,and little has been done to compare them with subjective data that contains a wide variety of image modalities and fusion algorithms.So quality assessment metrics for MMIF images specifically designed are eagerly desired.A MMIF image database is first built employing the classical MMIF algorithms,20 radiologists participated in the subjective test,all with medical education background of imaging diagnosis related researches.For each image set,the radiologists were asked to give a ranking score between 1 and 5 to each fused image within a continuous range,which can obtain more accurate subjective evaluation,where 1 denotes the worst quality and 5 the best.Subjective mean opinion score(MOS)is as ground truth.Then,phase congruency and standard deviation are combined to acquire the overall quality score.Finally,comparative study experiments are implemented based on the medical image fusion database.Experimental results reveal that the proposed algorithm can maintain good consistency with the subjective experimental results,which is superior to the existing state-of-the-art metrics,and is more applicable to evaluate MMIF images.(4)Multimodal Medical Image Fusion Algorithm Optimization Based on Visual Quality Assessment is proposed in this dissertation.In simplified PCNN,many important parameters are optimized manually or through a lot of training.So that it's difficult to get satisfying effect which limits in dealing with medical images with different modalities.This paper presents PCNN optimized by multi-swarm fruit fly optimization algorithm(MFOA)for multimodal medical image fusion.In the search iteration process,because only the fitness function needs to be used to evaluate the current position,the design of the fitness function is the key.Establishing a visual quality evaluation standard that is consistent with the subjective perception of the doctor is the judgment basis and the optimization goal.To increase the efficiency and quality of MFOA,the proposed MMIF quality evaluation measure was chosen to be the fitness function.Guided by such visual quality assessment,the performance evaluation of the fusion algorithm and the dynamic optimization of the parameters are adjusted through the evaluation results,and the optimal variables can be automatically matched with the source image to obtain the optimal parameter combination,enhance the fusion effect,and realize the algorithm optimization for MMIF.Experimental results visually and quantitatively show that the proposed fusion strategy is more effective than state-of-the-art methods and it is more effective in processing medical images with different modalities. |
PDF Full Text Request