Deep Learning For Glioma Whole Slide Image Analysis

Objective:Glioma is the most common malignant tumor in the central nervous system among adults.Although it only accounts for less than 1%of the total cancer incidence,malignant glioma has a high mortality rate.Patients with malignant glioma have poor prognosis even treated with the standard plan due to the biological characteristics and the site of the tumor.Surgical treatment is the cornerstone of glioma treatment.The pathological examination based on pathological images after surgery is also crucial in the treatment of malignant glioma.Pathological diagnosis of glioma is difficult to make.The histomorphologydependent diagnostic approach is highly reliant on the pathologist’s experience and subjective judgment.Research on molecular mechanisms of tumors has progressed rapidly in recent years.The WHO Classification for Tumors of the Central Nervous System,issued in 2016,also included molecular biomarkers in the diagnosis and classification criteria for glioma.These changes had made the diagnosis of glioma more specialized.The cost of diagnosis had also increased.The rapid development of deep learning technology has made using deep learning method to analyze medical images one of the most trending research directions.The whole slide image,a digital storage form of slides,enables the use of deep learning for pathology image analysis.Given the diagnosis of glioma pathological images a difficult,high-cost,and time-consuming job,there is huge potential for using deep learning method to develop diagnostic models to assist pathologists.The study aims to train deep neural networks capable of classifying glioma pathological images.Materials and Methods:This study has three sections.Section One:We collected all FFPE WSIs of glioma in the TCGA database and revised their diagnosis.We also collected pathology reports of glioma from our hospital between year 2016-2018,and preliminarily established a database,from which we extracted and scanned some sections.We also tried loading WSI with a library named OpenSlide,removing background area in WSI by several methods,segmenting WSI with DeepZoom module,and normalizing the color of WSI using color space migration method in Python environment.Section Two:In this section,data from the TCGA database were used to compose the dataset,while models using deep neural network designed to be able to classify histological and molecular pathological indicators for glioma tiles were constructed,and these models were trained,validated,and tested under supervised learning.A total of 28000 tiles of high-grade glioma and lower-grade gliomas,27886 tiles of IDH mutant and wild-type GBM,and 26384 tiles of TERT mutant and wild-type GBM were included to construct the dataset.The CNNs we used were mainly based on InceptionV3 or ResNet-50.The pathological diagnosis given by the pathologist and the results of molecular indicators obtained by sequencing were used as the evaluation criteria.The diagnostic performance was evaluated by accuracy and loss.Section Three:In this section we used attention mechanism and transfer learning to try to construct a new deep neural network model,and trained this exact model only using the final diagnosis of the WSI as labels under weakly supervised learning.This part of the study used TCGA data for model training and validation.The external validation was performed with data from our hospital.The WHO high-level and lowerlevel dichotomous task included 1237 WSIs from TCGA database and 436 WSIs from our hospital.The task of grading astrocytic tumors included 959 WSIs from TCGA database and 378 WSIs from our hospital.The task of grading oligodendroglial tumors included 278 WSIs from TCGA database and 56 WSIs from our hospital.The task of distinguishing astrocytic and oligodendroglial tumors included 1255 WSIs from TCGA database and 434 WSIs from our hospital.The task of classifying IDH status of astrocytic tumors included 897 WSIs from TCGA database.The attention mechanism was used when constructing the neural network in this section.Also,the pathological diagnosis given by the pathologist and the results of molecular indicators obtained by sequencing were used as the evaluation criteria.The diagnostic performance was evaluated by AUC and accuracy under 10-fold validation.Results:Section One:760 WSIs in the TCGA-GBM dataset and 844 WSIs in the TCGALGG dataset were collected.All outdated diagnosis were corrected in accordance with the WHO guideline published in 2016.A total of 2915 pathology reports from our hospital between 2016 and 2018 were collected and collated.All those reports were organized according to clinical information,WHO grade,final diagnosis,and molecular biomarkers.A total of 349 GBM slides,33 diffuse midline glioma slides,and 90 slides of grade Ⅱ and Ⅲ glioma were scanned.Threshold method,Otsu method,region growing and Grab Cut method were all able to remove the background of WSI.The color space migration method could accomplish color normalization of pathology images.A complete WSI pre-processing workflow was established by the above steps.Section Two:We successfully constructed models that could classify histological and molecular indicators for glioma tiles using classical convolutional neural network structures,and the training process was performed only with H&E-stained images.The model using ResNet-50 architecture achieved 90.78%accuracy with a loss of 0.3010 on the WHO grading task.The model using InceptionV3 architecture achieved 92.76%accuracy with a loss of 0.1904 on the task of differentiating IDH.The model using InceptionV3 architecture achieved 93.56%accuracy with a loss of 0.1705 on the task of differentiating TERT promotor.The three models could complement each other in predicting glioma tiles for different task objectives.Section Three:We successfully constructed a new weakly supervised learning model and completed the analysis or prediction of glioma WSIs,with the training process using only H&E-stained images.The average AUC for grading of glioma,grading of astrocytic tumor,grading of oligodendroglial tumor,distinguishing the origin of glioma,and distinguishing IDH status of astrocytic tumor were 0.9904,0.9419,0.8659,0.9298,and 0.9488.The average accuracy were 94.39%,82.21%,81.11%,88.36%,and 89.31%.The accuracy on external validation were 91.47%,76.31%,67.59%,and 84.17%.Once the results were obtained,a corresponding heat map was also drawn based on the confidence score generated by the model during prediction procedure,helping pathologists to quickly target the area of interest.Conclusion:In this study,datasets of WSIs were constructed using the TCGA database and data from our hospital.We have also established a pre-processing workflow pipeline of WSI,which was loading-background removal-segmentation-color normalization.Under supervised learning,we applied deep convolutional neural networks to establish analysis models of glioma tiles,and completed the WHO grading analysis task.We also tried to establish deep learning models for molecular indicators,which were less involved in previous studies,and completed the prediction of two molecular indicators of glioma,IDH and TERT.Under weakly supervised learning,we proposed a new deep neural network model based on attention mechanism and transfer learning.The model can be trained using only H&E-stained images and with a small sample size,and can directly analyze the entire glioma WSI,maintaining a high level of accuracy for several tasks.The heat maps drawn based on confidence score were able to help the pathologists to quickly target the region of interest on the entire slide.