Predicting Treatment Response To Neoadjuvant Chemoradiotherapy In Rectal Cancer Based On Deep Learning

The standard treatment for locally advanced rectal cancer is neoadjuvant chemoradiotherapy combined with surgical resection.However,neoadjuvant chemoradiotherapy may have additional toxicity.Pathological biopsy is the "gold standard" for clinical diagnosis and treatment,and contains a large amount of tumor microenvironment information,but doctors cannot obtain the treatment response of neoadjuvant chemoradiotherapy for patients from clinical diagnosis.Therefore,this paper proposes to biopsy digital pathology images combined with deep learning methods to predict the treatment response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer,to assist doctors in clinical decision-making.Due to the large size of digital pathology images and the difficulty of labeling,we based on deep convolutional networks and small patches of digital pathology slice,conducts research from two aspects: unsupervised cell nucleus segmentation methods and neoadjuvant chemoradiotherapy treatment response prediction methods.The main research of this paper is as follows.(1)This paper improves the original weakly supervised cell nucleus segmentation method,and designs an unsupervised cell nucleus segmentation method,which is divided into unsupervised nucleus detection network and nucleus segmentation network.This paper designs an unsupervised nuclear detection model based on the characteristics of hematoxylin and eosin stained pathology slides.The label of cell nucleus segmentation is generated from the nuclear center detection point generated by the detection model for the training of the segmentation network.In addition,this paper designs a dynamic correction strategy that uses the predict map of the nucleus segmentation network to continuously iteratively correct the nucleus detection model,and continuously optimizes the detection model to improve the performance of the segmentation model.By using the dynamic correction strategy,the Aggregated Jaccard Index is increased by 13.2% compared with the Aggregated Jaccard Index without the dynamic correction strategy.(2)This paper proposes a multi-scale prediction network based on auto-encoder and adversarial generation network.In this paper,the cell nucleus segmentation network is used to select patches with the highest cell density for subsequent analysis.Aiming at the possible domain differences between natural images and pathological slides,this paper combines the auto-encoder and the adversarial generation network to design a feature extraction module to extract more abstract and efficient image features.And this article uses patches of pathology slides with different magnifications to design a multi-scale prediction module based on attention mechanism,which effectively uses the tissue information and detailed information presented under different magnifications in histopathological slices.Through experimental verification,our proposed method benefits from the various components proposed in this article,and is superior in performance to other proposed methods,and has the potential to predict the treatment response of locally advanced rectal cancer after neoadjuvant chemoradiotherapy.