Feature Extraction And Verification Of Pulmonary Infectious Diseases Based On Deep Multi-modal Learning

Objective: To study and establish an inference model for the auxiliary diagnosis of infectious respiratory diseases based on multi-modal deep feature fusion,and establish a multi-modal database combining imaging,laboratory examination results and clinical manifestations for infectious respiratory diseases.Methods: The NGS data,clinical manifestations,laboratory test results,imaging test results and other multi-modal data of patients with infectious respiratory diseases were effectively collected and missing values processed.It also studies the deep feature fusion algorithm of multimodal data,couples the private and shared features of different modal data of infectious respiratory diseases,and digs into the hidden information of different modalities to obtain efficient and robust shared features that are conducive to auxiliary diagnosis..In view of the fact that there is no unified standard for the scope of data retrieval and database establishment of existing infectious respiratory disease cases,through the compilation of existing retrospective data and the follow-up of historical data,a large number of new infectious cases and their derivatives are collected and tested Complete high-throughput genomics data and clinical association data of pathogenic microorganisms,formulate data retrieval range,and summarize case data.Aiming at the problems of data missing and inaccurate data in aggregated multi-modal data,the incomplete data filling algorithm based on distributed subtraction clustering is studied.The incomplete data is clustered by an improved subtractive clustering algorithm,and then the incomplete data is filled with the clustering result and weighted distance.Thereby,the data with missing attribute values can be filled in quickly and accurately,so as to prepare for subsequent tasks such as data mining and analysis.Select and compare commonly used models and algorithms,compare methods such as deep learning models,deep reinforcement learning models,and deep Q networks,and conduct pre-experiments and model training.Unsupervised multimodal data deep non-negative correlation feature fusion algorithm.By first constructing two projection matrices for each modal,the original data features are converted into modal private and modal shared feature representations.Then,an error function is constructed for each mode through the coupling of shared features between modes,and is jointly optimized.In this way,the modal private(irrelevant or negatively related)features and the related features between the modalities can be learned together to obtain more effective and robust shared features of multi-modal data in the latent subspace.Through the mutual supplement and sharing of knowledge between modalities,the hidden semantic information of multi-modal data can be mined to better assist the diagnosis of diseases.In view of the unbalanced data distribution among the different source data of the existing infectious respiratory disease cases,the data of some modal cases is relatively small,while the data of related modal cases is relatively large.In order to transfer sufficient knowledge of similar modalities to In modalities with few data instances,support the realization of target tasks,and study the deep migration feature fusion algorithm of unbalanced multi-modal data.Based on the high-level semantic abstraction characteristics of deep learning networks,a multi-layer semantic matching deep network architecture that couples modal deep networks and modal features is designed.Through multi-layer cross-modal feature correlation matching,the semantic deviation between heterogeneous modalities is gradually reduced.Use the maximum correlation of the top-level output features to optimize and adjust the overall modal network to further improve the relevance of the modal depth semantics.A new objective function is defined to jointly optimize the heterogeneous modal deep matching network,to obtain a cross-modal high-level semantic fusion subspace,and complete the transfer learning from the source modal knowledge to the target modal task in the subspace.According to the application requirements of the health care of infectious respiratory diseases,combined with basic data such as medical dictionaries,electronic medical records,various medical guidelines,and expert consensus,establish a knowledge map of infectious respiratory diseases.In a top-down approach,domain experts first construct an ontology database,and then extract relevant rules based on relationships,and populate the knowledge graph from heterogeneous data sources such as relational databases to form a domain knowledge graph.Results: Established a whole-process auxiliary diagnosis and reasoning model for infectious respiratory diseases.First,data preprocessing is carried out through relation embedding and entity embedding.The features learned through the deep model are used for feature fusion learning,and finally the disease state is predicted,and a complete set of diagnostic reasoning models for the entire process of infectious respiratory diseases are constructed.Conclusion: Through deep multi-modal feature extraction,the diagnosis and treatment model of lung infectious diseases can be established.This is the future research direction of the combination of clinical medicine and artificial intelligence.