Mutimodal Medical Image Mapping Based On Descriptors And Feature Matching

Multimodal Medical image mapping is that using data acquired by one imaging medical device predict data acquired by another different imaging device or parameter.Predicting Computerized tomography(CT)images from Magnetic Resonance(MR)images is a very important issue for multimodal medical image mapping.Synthetic pseudo CT images can provide the electron density information that required for MR-based radiotherapy planning dose calculations and PET attenuation correction,and reduce the damage that CT imaging ionizing radiation causes to patients.Due to the different imaging theroy between MR images and CT images,the mapping relationship between different modality images is very complicated.In order to solve the optimization problem of MR linear descriptor,we propose the learning nonlinear descriptors for MR images.In order to solve the problem of too long running time for predicting pseudo-CT images based on the descriptor learning algorithm,we propose an improved neighborhood anchor regression.The main research contents of this paper are as follows:(1)We propose an algorithm based on linear descriptor learning for the prediction of pseudo CT images.For the problem of insufficient MR feature information extracted by voxel values,original image patches or multi-scale image patches,we use dense scale-invariant feature transform algorithm and improved supervised descriptors learning(ISDL).This algorithm extracts the compact descriptors of MR images with CT manifold regularization,the structural information of the large spatial support,and the context information.In this experiment,the mean absolute error(MAE)between the real CT and pseudo CT images of 13 patients was 78.90±23.25 HU,and the peak signal-to-noise ratio(PSNR)was 30.40 ± 1.93 dB.The results show that the proposed linear descriptors based method can obtain pseudo CT images with high precision.(2)We propose an algorithm based on feature matching with learning nonlinear descriptors for the prediction of pseudo CT images.For the problem that the MR linear descriptor combined with the CT similar matrix are added to the ISDL algorithm directly to obtain the suboptimal result,and the probem that the performance of the pseudo CT images that applied to the PET attenuation correction and MR-based radiotherapy planning is untested.In this paper,the extracted linear descriptors of MR images are projected onto the high-dimensional kernel space through feature mapping.The linear descriptors are first extended to nonlinearity,and then the discriminative descriptors are learned by the ISDL algorithm.The performance of PET attenuation correction is verified by simulating the missing PET data.The mean MAE between the real CT and the pseudo CT images of 13 patients was 75.25 ± 18.05 HU,the PSNR was 30.87 ± 1.15 dB,the attenuation correction error(rMAE)was 1.56 ± 0.50%,and the radiotherapy dose calculation error was 0.055 ± 0.107%.The results show that the pseudo-CT images predicted by the proposed FMLND method are suitable for PET attenuation correction and MR-based radiotherapy.(3)We propose an algorithm based on Improved Neighborhood Anchor regression(INAR)for the prediction of pseudo CT images.TIn order to solve the problem that the too long running time of using the sliding window to search the neighbor samples in the nonlinear descriptors with feature matching method,this paper proposes a hierarchical search INAR method to pre-train the projected matrix,and through strategies including the augmentation of MR-CT dataset,data-driven optimization(INAR-O)and multi-regressor ensemble(INAR-E)to improve the accuracy of predicted pseudo CT images.We propose a revision of the measurement methods.The mean MAE between the real CT and the pseudo CT images of 22 patients was 92.73 ±14.86 HU,the PSNR was 29.77±1.63 dB,and the rMAE was 1.30±0.20%.The remaining 15 patients whose projected matrices were trained using only 7 patient achieve an average MAE of 106.89±14.43 HU and an rMAE of 1.51±0.21%.The experimental results show that the proposed INAR-E method can not only predict pseudo-CT images with little time,but also obtains considerable pseudo-CT images with limited data.In this paper,the proposed three methods can gradually improve the performance of predicting pseudo-CT images.The experimental results demonstrated that the predicted pseudo-CT images for PET attenuation correction and dose calculation have low errors.