GAN-Based Image Generation In The Early Stage Of Neoadjuvant Chemotherapy And Response Prediction In Breast Cancer

Breast cancer is a serious threat to women’s health,and neoadjuvant chemotherapy is an important means of treating breast cancer.Neoadjuvant chemotherapy usually consists of multiple courses,with a long treatment period and varying response among individuals.Some patients respond well to treatment,while others may experience disease progression due to poor response,missing the optimal window for receiving other treatment options.If the patient’s response to chemotherapy after treatment or the trend of changes in the lesion during chemotherapy can be accurately predicted before treatment,it would have important clinical significance.Dynamic Enhanced Magnetic Resonance Imaging(DCE-MRI)is a common method in the diagnosis of breast cancer.Radiomics research is of great significance to construct response prediction models by extracting imaging features to assist doctors in clinical decision-making.The existing methods are mostly based on DCE-MRI with a single collection,but the actual performance is not good.Some studies combined DCE-MRI with two collections before and after the first chemotherapy course to predict the response,but the model is only available for patients who have undergone one treatment,which has limitations.We conducted research on early chemotherapy stage image generation and final response prediction using two sets of DCE-MRI data collected from breast cancer patients before and after the first course of neoadjuvant chemotherapy,along with their final response data.A novel GAN-based response prediction model was designed,using early stage DCE-MRI of chemotherapy as training constraints.This model can generate early stage DCE-MRI of chemotherapy using pre-chemotherapy images,thereby eliminating the need for actual imaging after one treatment to observe treatment progress.The model was also trained through a generative adversarial process to constrain the response prediction learning direction,resulting in more accurate response predictions.The model can directly predict and generate patient final response and early stage DCE-MRI using prechemotherapy images.Doctors and patients can use the generated images to visually observe the possible changes in the lesion after the first chemotherapy course and to formulate more accurate treatment plans based on response prediction information.This thesis mainly includes the following research contents:(1)Research on response prediction based on CNN and design of perceptual loss.In this thesis,we first conducted a baseline study on response prediction based on pre-chemotherapy DCEMRI,and designed a perceptual loss for image generation research in subsequent sections.We used various CNN structures to design response prediction models,and introduced attention mechanisms to improve the performance of prediction.The results showed that the best AUC and accuracy of the CNN response prediction model were 0.78 and 0.72,respectively.Due to the difference in breast morphology between the pre-and post-treatment images of patients,the generation model constrained by the classical distance loss have poor performance.In addition,the original perceptual loss used Image Net pre-trained models as features extractor models,but there are differences in features between DCE-MRI and natural images.Therefore,in this thesis,we designed the perceptual loss based on the CNN response prediction model of DCE-MRI.(2)Research on early-stage chemotherapy image generation based on GAN.We conducted research on DCE-MRI generation in the early stage of chemotherapy based on GAN.By using GAN to generate images after early-stage chemotherapy from patient images before treatment,we attempt to learn the pattern of changes in breast lesions during chemotherapy from the connection and difference between the two sets of DCE-MRI.We adopted two generation strategies,pix2 pix GAN and Cycle GAN,and introduced an attention mechanism to optimize the generator network.The perceptual loss designed in the previous section was used to constrain the deep feature similarity between the generated images and the real post-chemotherapy images.Experimental results showed that the Cycle GAN model with attention mechanism and perceptual loss performs the best,with an SSIM mean value of 0.90 for image generation indicators,an FID of 19.83 for model evaluation indicators,and good model generation performance.(3)Research on response prediction of deep learning models based on early-stage chemotherapy image constraints.We constructed a multi-modal prediction model by embedding a classifier into GAN,which can simultaneously predict and generate the final response and early-stage chemotherapy DCE-MRI for patients.By combining the two,we expect to promote the model to learn the changing trend of breast lesions during chemotherapy by introducing early-stage chemotherapy images as constraints in response prediction training,thereby accurately predicting response.In addition,response information was added as a constraint in image generation training to make the generated results closer to real response images.The results showed that the model generated post-chemotherapy image indicators with an SSIM mean value of 0.92 and an FID of 19.33.Using pre-chemotherapy images to predict response,the best AUC and accuracy in the test set were0.86 and 0.80,respectively.The performance of this model in image generation and response prediction is significantly improved compared to the above single-task models.Finally,we verified the effectiveness of the model through ablation experiments and conducted interpretability research on the model using class activation maps.We proposed a novel GAN model-based generation of DCE-MRI image after early treatment of neoadjuvant chemotherapy and final response prediction method for breast cancer.Compared with the traditional GAN method for medical image generation in the same time state,we extended the GAN method to the field of cross-time medical image generation.Compared with the traditional response prediction method,we introduced the prior information of the images in the early stage of chemotherapy in the training process,which improved the performance of response prediction.This research has important clinical value and significance for assisting doctors in clinical decision-making through technical means and formulating individualized diagnosis and treatment plans for different breast cancer patients.