Prediction Of Breast Cancer Neoadjuvant Chemotherapy Based On Longitudinal Time Depth Network Fusion

Breast cancer is the cancer with the highest proportion of all female cancer patients.Neoadjuvant chemotherapy is a common method for the treatment of breast cancer.It mainly uses preoperative chemotherapy to reduce the size of the tumor and facilitate subsequent surgical resection,thereby improving the therapeutic effect of breast cancer,has been widely used.However,in some patients,the tumor did not shrink after chemotherapy,or even expanded further,thus delaying the optimal treatment time.Therefore,it is extremely important to predict the efficacy of the final course of neoadjuvant chemotherapy.Neoadjuvant chemotherapy usually consists of 6 to 8courses.According to the changes in the shape and size of the tumor mass after early chemotherapy(2 courses),it can provide reference value for patients to adjust the chemotherapy regimen.Therefore,it is of great significance to predict the efficacy of final chemotherapy according to the characteristics of tumor before and after early chemotherapy.Medical imaging is a common method for diagnosing breast cancer and has been widely used in clinical diagnosis.Many studies have shown that breast DCE-MRI can effectively evaluate the efficacy of neoadjuvant chemotherapy.In clinical diagnosis,image analysis only based on the doctor’s experience and subjective judgment often cannot fully and accurately evaluate the pathological information.With the rapid development of medical imaging technology and the continuous updating of image recognition technology and algorithms,big data mining and analysis of medical images can effectively improve the accuracy of clinical diagnosis and treatment.In this study,DCE-MRI images of breast cancer patients were collected from Zhejiang cancer hospital and Fudan Cancer Hospital,and the pathological information of patients,including surgery information,chemotherapy course,final efficacy and other information,were statistically analyzed according to the pathological report.The breast region was segmented in the images before chemotherapy and after early chemotherapy.The preprocessed image data is enhanced to meet the needs of the experiment.In this paper,the final efficacy of neoadjuvant chemotherapy is predicted by feature level fusion and decision-making level fusion according to the longitudinal time images of patients,and two attention mechanisms are further introduced on the basis of the prediction model integrating the two network structures.According to the existing multiple network structures,multiple chemotherapy efficacy prediction models are built by improving its full connection layer,and ablation experiments are carried out on multiple prediction models.Firstly,the pre chemotherapy images and the early post chemotherapy images are input into the prediction models constructed by different neural networks,the receiver operating characteristic curve(ROC)is drawn,and the area under the ROC(AUC)is calculated to compare and analyze the prediction results.The experimental results showed that the best AUC for predicting the final efficacy using pre chemotherapy images was 0.808,while the best AUC for predicting the final efficacy using early post chemotherapy images was 0.821,and the prediction results using early post chemotherapy images in multiple prediction models were better than those using pre chemotherapy images.In order to verify the effectiveness of the neural network used in the experiment for the final efficacy prediction,ablation experiments were carried out on the efficacy prediction models based on longitudinal time images.The results show that the prediction models built by several neural networks used in this study can effectively predict the final chemotherapy efficacy.In order to fuse pre chemotherapy images and early post chemotherapy images,model fusion methods are designed based on feature level and decision level respectively,and attention mechanism is added to improve model performance.Based on the feature level,this paper designs a curative effect prediction model that integrates the two network structures.Based on the decision level,the weighted method is used to give the corresponding weights to the pre trained prediction models,and multiple models are fused for comprehensive decision-making.The experimental results show that the optimal AUC of the model prediction results based on feature level fusion is0.863,while the optimal AUC of the model prediction results based on decision level fusion is0.829.The performance of using pre chemotherapy images and early post chemotherapy images to predict the final efficacy of neoadjuvant chemotherapy is better than using pre chemotherapy images or early post chemotherapy images alone.In order to further improve the effect of the final efficacy prediction,this paper introduces two attention mechanisms based on the prediction model of the fusion of the two neural networks,adds the channel attention module and the spatial attention module,assigns different weights to different channels and different spaces of the input image,suppresses the information that is invalid for the prediction results,and enhances the information that is valid for the prediction results.The results show that the best AUC of the final efficacy prediction model after introducing attention mechanism is 0.922,so adding attention mechanism can further improve the prediction performance of the model.