Cross-Modal Medical Image Transformation Based On Multiple GANs

Magnetic Resonance Imaging(MRI)technique can obtain multi-modal medical images,such as T1-weighted,T2-weighted and FLAIR images.The multi-modal medical images reflect different tissue information of human organs and are widely used in clinical treatment of pathological detection and clinical treatment.However,the limitations of imaging principles have led to expensive costs and a long-time process for obtaining multiple modal MRI images.Therefore,the estimation of desired modal images from acquired modal images without real acquisition can reduce costs and improve diagnosis efficiency,such as the prediction of missing T2-weighted images from acquired T1-weighted images.Aiming at the problem that current methods cannot guarantee the consistency of pathological information between predicted images and target images during prediction process,and can only predict images in specific unidirectional fashion after one training process.This thesis proposes a cross-modal medical image bidirectional transformation method based on multiple generative adversarial networks and a multiple modal medical image multi-directional prediction method based on generative adversarial network,which are used to maintain the pathological information invariance of source image and complete multi-direction prediction between multi-modal MRI images.The research works of this thesis are listed as following:1.In order to solve the problem that current prediction methods can only predict images in specific unidirectional fashion after one training process,and cannot preserve the pathological information of input image does not change during prediction process.This thesis proposes a cross-modal medical image bidirectional transformation method based on multiple generative adversarial networks,which combines deep feature losses and handcrafted feature losses and learns bidirectional non-linear mapping between two modal MRI images,such as T1-weighted and T2-weighted images.This algorithm introduces pathological labels to maintain the consistency of pathological information.The experimental results on paired datasets and unpaired datasets show that this algorithm can complete cross-modal bidirectional prediction after one training while ensuring the invariance of pathological information between predicted images and source images.2.Aiming at the problem that current unidirectional and bidirectional prediction methods cannot complete multi-directional prediction between three or more modal MRI images,such as completing T1-weighted to T2-weighted and T2-weighted to FLAIR images prediction after one training,and fully using a large number of unpaired datasets to training.This thesis proposes a multi-modal medical image multi-directional prediction method based on generative adversarial network,which introduces a modal encoder to constrain the consistency between target modal codes and the modal codes of predicted images.This thesis can accomplish the multi-directional prediction of three or more modal MRI images and fully use unpaired datasets for training.The experimental results in supervised and unsupervised fashion show that this method can complete multi-modal MRI image multi-directional prediction and obtain more accurate predicted images in unsupervised fashion compare with comparison methods.3.Considering the problems of current deep learning programs are all running in the background and doctors cannot easily use it.This thesis designs and implements a cross-modal medical image prediction system for multi-modal MRI images.This system can automatically preprocess input MRI images for suiting the input format of cross-modal prediction model and set the input and output parameters of prediction model,and exporting predicted images after transformation.The user can intuitively select input MRI images and prediction model through system interface,and this system will automatically output the desired target modal images after setting corresponding parameters.This system is not only convenient to use,but also can promote the implementation of cross-modal medical image prediction algorithms.