Research On Multi-modality Medical Data Representation Learning Technology Based On Deep Network

Representation learning techniques refer to a range of techniques that transform the original data distribution according to the task into what the machine can effectively recognize and apply.As an effective technique for representation learning,deep networks have been extensively studied in computer-aided diagnosis systems through the accurate representation of disease heterogeneity,which has promoted the development of intelligent healthcare.But unlike the natural images,the heterogeneous features of lesions usually present a multimodal distribution in medical clinical data,so that the representation learning needs to consider more complex data environments to effectively improve the accuracy of computer-aided diagnosis.Therefore,the study of multi-modal data representation learning technology based on deep network can promote the computer-aided diagnosis system based on multi-modal data environment to exert greater benefits in intel igent medical treatment.Aiming at the application problems of multi-modal medical data in the three phases of feature extraction,feature fusion,and model prediction,this paper respectively focuses on the three aspects of fine-grained,interdependent,and robust representation learning.For the predicted performance bottleneck of medical images based on the representation of deep network,the research on fine-grained representation improves the representation learning of image;for the complementarity of non-image and image data in disease representation,the research on the fusion of interdependent representation improves the comprehensive representation performance of diseases;in view of the differences in the representation of multi-center images,the study of robust representation problem further expands the application scope of image representation learning.Three aspects of research have improved the representation learning performance based on multimodal medical data.The main research results are as follows:1.This paper proposes a multi-modal image fine-grained representation learning method by combining spatial and channel attention mechanism.Through summarizing the assumptions of channel and spatial attention mechanism that the redundant representation is in terms of the feature type and image background,this paper fuse two dimensions of space and channel attention mechanism through the different feature extraction pathways in convolutional networks and finally improve the performance of fine-grained representation of CT and PET images.In the PET / CT lung cancer experiment,comparing the different fine-grained characterization methods,the performance of the multi-dimensional attention mechanism has been improved by at least 5%.The highest AUC(Area Under Curve)reached 85.5%,indicating that the representation learning method based on multi-dimensional attention can effectively characterize the heterogeneity of the lesion under different image modallity.Combined with the visual comparison of class activation map(CAM),the complementarity of the attention mechanism of the two dimensions is demonstrated.2.A method to fuse multi-type interdependent feature based on long-short term memory network(LSTM)is proposed.Considering the LSTM advantage on sequence relation representation,this paper design an interdependent feature fusion method based on LSTM to fuse the common representation of non-image and image data for disease and overcome the defects of the fusion of multi-type feature and the multi-collinearity problem.In the EGFR classification experiment based on CT images and clinical features,this method increased the AUC value to 78.00%,which was 5% higher than other methods,and verified the improvement of the representation performance by model interdependence based on LSTM.In addition,through the unsupervised clustering analysis of feature space,the Randall adjustment coefficient reached 0.93,which verified the distinguishing effect of interdependent features on spatial distance.3.A multi-center image robust representation model based on domain confounding representations and cycle-consistency of classification is proposed.In order to achieve the constraints of robust representation between multi-center images in the process of migration prediction,this paper introduces cross-domain image confusion training by constructing a paired GAN Generative Adversarial Networks(Generative Adversarial Networks)network to achieve the representation of the domain-invariant space;Combined with the cycle-consistency of classification,the domain-independent representation is further focused on the representation of the category information under the complex feature distribution,so as to achieve the robust representation of the classification information.In the experimental results of the migration prediction of benign and malignant lymph node in plain CT and enhanced CT scans,the accuracy reached 73.8%,which was at least 4.40% higher than other method,and the effective migration prediction between multi-center images of lymph nodes was achieved.