CT/MRI Multi-modality Image Synthesis And Transformation Based On Deep Learning

In radiotherapy,the general way is using computed tomography(CT)to acquire a set of planning CT images before radiotherapy.Since the planning CT images have the electricity density information,which can be used to calculate the radiation dose.With the development of radiotherapy technology,magnetic resonance image-guided radio-therapy(MRIgRT)becomes more and more popular.However,there is no electricity density information in MRI images.To solve this problem,in this paper,we constructed a neural network model which named RUN based on U-Net and Res-Net.Then using this deep learning model to synthesize CT images from corresponding MRI images by inputting a lot of training MRI and CT data to learn the mapping from MRI to CT.Thus computing radiation dose according the synthetic CT images.Experiment demonstrate that our RUN model can effectively convert MRI images to corresponding real-like CT images.Furthermore,RUN converges faster and needs less model training time when compared with other models.In MRIgRT system,traditional cone beam computed tomography(CBCT)imag-ing is replaced by MR imaging.To decrease patient set-up error,MRI images are used to register with planning CT.However,cross modality registration between MRI and CT images is not as easy as mono-modality registration.To address this issue,we constructed three deep neural networks,including U-Net,Pix2Pix and CycleGAN.We input the constructed networks with MRI and CT images to learn the CT to MRI mapping.Then using the well trained network to convert CT images to correspond?ing MRI images.The synthetic MRI images can be used to register with positioning MRI images.Thus,the CT-MRI cross modality registration problem becomes MRI-MRI mono-modality alignment issue to improve the registration accuracy.In addition,we designed experiments to further analysised the model performance between super-vised learning and unsupervised learning.Experiments show that supervised based U-Net outperformed unsupervised CycleGAN.Furthermore,the best performanced U-Net model was optimized to further improve its performance.In this paper,by using deep learning and combining with the idea of multi-modality medical image transformation,we designed experiments aiming at solving two prob-lems:MRI image based radiotherapy dose calculation and improving patient position-ing accuracy in MRIgRT.We constructed neural network models and verified the va-lidity of these models by experiments.