Unpaired Multi-Contrast Magnetic Resonance Image Synthesis Using Generative Adversarial Networks

Magnetic Resonance Imaging has emerged as one of the most dominant imaging techniques in radiology for its non-invasive and multi-contrast nature.It has become the first choice of physicians for the detection of pathology due to its high resolution and detailed scans.These multi-contrast MR images are of great importance for deep learning methods as well in tumor detection and white/gray matter segmentation.However,it is quite difficult to acquire multiple contrasts of the same patient due to cost,time and motion artifacts problem.This missing contrast issue causes difficulties for deep learning methods as well as for physicians in better diagnosis.Therefore,it is of great importance to synthetically generate missing MRI contrasts from the available one.Generative Adversarial Networks（GANs）have derived many impressive advances in the direction of medical image synthesis.However,existing GANs based methods for medical image synthesis are limited by the design and can only learn to generate one type of contrast from the other in a one-to-one manner.This structural limitation makes these methods inefficient for multi-contrast MR synthesis as multiple models are needed to be trained separately.Also,the majority of these methods require paired data for training which is extremely challenging to acquire for multi-contrast synthesis.Although some methods can work with unpaired data but are still not effective in utilizing the data available for all contrasts.This inefficiency and ineffectiveness of existing methods result in massive training time and a huge waste of computational resources.To overcome the training of additional models for multi-contrast synthesis,we propose a novel Perceptual StarGAN for the unpaired multi-contrast synthesis of MR images.Perceptual StarGAN utilizes a single generator network to learn multi-contrast synthesis without separate training for each pair of contrasts.To enhance the synthesis quality of our method we utilize a U-Net based generator which has reportedly well with medical images.To focus more on small details which are quite essential in medical images we downsample the image 7 times for generator and discriminator network pushing our model to focus on small anatomical details of each contrast.Also,we propose a new loss function for the generator network,which penalizes the generator for missing small structural details.This new generation loss is also sensitive to the perceptual difference between real and synthetic images,the generator is trained to output image showing high structural and perceptual similarity with real images.We show the synthesis results on four common MRI contrasts including T₁-weighted,T₂-weighted,PD-weighted and MRA provided by public IXI-dataset.To make our method more applicable,we propose an efficient architecture that reduces the training time of our Perceptual StarGAN.We name this new architecture as Efficient Perceptual StarGAN（EP-StarGAN）.This new architecture is specifically designed to keep our computational budget low to enhance the training speed.Instead of going deeper like Perceptual StarGAN,we introduce a new residual-inception module for generator and discriminator architecture which captures the non-linearity in data on a small patch level scale.EP-StarGAN reduces the training time for multi-contrast MR synthesis by 4 times and increases the quality as well.Detailed qualitative and quantitative results for Perceptual StarGAN and EP-StarGAN show the superiority of our approach for multi-contrast MR image synthesis with unpaired data.