Study On Breast Cancer Survival Prediction By Integrating Gene Expression Data And Pathological Images

The most common malignancy among females is breast cancer,which is one of the leading causes of cancer-related deaths in the world.Accurate survival prediction of breast cancer is important for assisting clinicians to prescribe the most appropriate therapeutic regime and helping establish palliative care and hospice care system.Previous studies have shown that both gene expression data and pathological images contain complementary information that is related to breast cancer survival prediction.Therefore,it is of great significance to effectively combine gene expression data and pathological images and further improve the performance of breast cancer survival prediction.However,most of existing cancer survival prediction methods directly concatenate gene expression data and pathological images,which may ignore the intrinsic relationship of the features across different modalities.In order to solve this problem,this study fully exploits intrinsic relationship of the features across different modalities and effectively captures interactions within each modality.The main contributions of this paper are as follows:(1)To effectively combine gene expression data and pathological images,a survival prediction method based on bilinear model,named GPDBN,is proposed.Firstly,CellProfiler is used to extract the nuclear,cytoplasmic and image level features from pathological images.Then,the inter-and intra-modality bilinear feature encoding modules in GPDBN are used to fully capture the interactions between and within different modalities.Finally,a deep prediction module is used to accurately predict the survival time of patients.The evaluation results show that both the inter-and intra-modality bilinear feature encoding modules in GPDBN can help to improve the performance of breast cancer survival prediction,and the performance comparison with existing methods also shows the superiority of GPDBN.(2)To fully exploit the survival related information contained in pathological images and make full use of the survival information contained in the censored patients,a deep bilinear survival prediction method,named DBNSurv,is further proposed.Firstly,VGG19 deep convolution neural network is used to extract the high-level features from pathological images.Then,inter-and intra-modality bilinear encoding modules followed by a deep feedforward neural network are used to effectively combine gene expression and pathological features.Finally,Cox partial log likelihood loss function is used to train the model.Experimental results show that VGG19 deep convolution neural network can significantly improve the performance of breast cancer survival prediction.Compared with existing methods,DBNSurv also achieves better performance of survival prediction.