The high incidence of malignant tumor seriously affects human health,and early detection of cancer can effectively reduce the mortality of malignant tumor.Due to the overall shortage of medical resources in China,artificial intelligence technology is needed to improve the diagnosis and treatment level and work efficiency of doctors.In recent years,tumor diagnosis technology based on deep learning has made great progress,but most methods can only deal with standard Euclidian data,while in the diagnosis and treatment process,doctors usually need to make comprehensive use of clinic pathological characteristics,imaging examination,serological examination,gene and other multi-modal data for accurate diagnosis of patients.Compared with other methods,graph convolutional neural network can effectively process multimodal data and has higher application value.However,the current graph convolutional neural network classification algorithm is still limited to the application of binary classification such as benign and malignant diagnosis of tumors,which cannot meet the actual clinical needs of fine staging and grading of malignant tumors.Therefore,this thesis firstly studies the graph convolutional neural network classification algorithm based on medical multimodal data,and on this basis,verifies the effectiveness of the design algorithm in tumor diagnosis task through simulation experiments.The main research contents of this thesis include:(1)To make full use of effective information in multi-modal data,the construction algorithm of the input graph is studied.The data included medical imaging data and non-imaging phenotypic data describing clinic pathological features.On the one hand,a population graph was created by screening suitable phenotypic information for non-imaging data such as age,sex,tumor morphology and tissue calcification.On the other hand,the feature difference between medical images is used to construct a dynamic learning graph,and the structure of the graph is dynamically optimized by the loss function.(2)In order to improve the performance of graph convolutional neural network classification algorithm,a graph convolutional neural network architecture that can fully mine the deep features of input graphs is constructed.By designing multi-layer information aggregation blocks to optimize the feature fusion process of the model,and providing sufficient neighborhood information for all nodes while suppressing excessive smoothing,the overall classification capability of the model was significantly improved.(3)In order to solve the clinical application problems of accurate tumor diagnosis,a general tumor diagnosis framework was designed by combining the proposed input graph construction algorithm and graph convolution neural network architecture.By comprehensively analyzing the feature extraction results of the two input images,the feature fusion ability of the model was improved,and the accuracy of benign and malignant tumor classification,clinical staging and grading was effectively improved.The experimental results demonstrate that the general tumor diagnosis framework proposed in this thesis can make full use of medical multi-modal data to solve clinical diagnosis problems,and has certain universality in multiple diseases.It has achieved 98.6% and 98.4% average classification accuracy in benign and malignant diagnosis tasks of breast cancer and autism spectrum disorder,respectively,and 91.7% average classification accuracy in BI-RADS classification tasks of breast cancer.Compared with previous deep learning methods,it has higher practical application value. |