Research On Design Of Multi-attention And Knowledge-assisted Neural Network For Intelligent TCM Aided Diagnosis And Treatment

Traditional Chinese medicine(TCM)plays a vital role in our country's medical system.For thousands of years,it has made great contributions to the prosperity of the nation and the health of the people.Especially in the recent new crown epidemic(COVID-19),traditional Chinese medicine diagnosis and treatment played a crucial role.Under the background of "artificial intelligence +",the research of intelligent Chinese medicine technology can open a new pattern of intelligent development for Chinese medicine.The knowledge system of traditional Chinese medicine is huge and complex,and physicians rely heavily on a large amount of clinical experience.It is an inevitable trend of intelligent TCM research and development to integrate the knowledge of TCM with multi-source body sign information,and then carry out efficient and objective healthcare management in the whole process.The research content of this thesis focuses on the consultation and inspection part of the intelligent TCM assisted diagnosis and treatment technology.Although the deep learning technology based on the neural network has been successfully applied to various domains including the medical field,it is similar to other intelligent medical applications,when we study the questioning and inspection tasks in the auxiliary diagnosis and treatment of intelligent TCM,we will face the following general or unique challenges: 1.The medical data is full of noise,and whether it is text data such as patient questions,or the image data of the patient's tongue,etc.,it is often the detailed features that have a powerful influence on the prediction results;2.The diagnosis and treatment task of TCM is a highly professional prediction task and requires rich background domain knowledge as a guide;3.The collection cost of TCM clinical data is high,the available data is limited,so the requirements for the modeling efficiency of the neural network model are higher than other tasks;4.The TCM clinic is mainly based on general medicine,and it is often necessary to deal with rare cases.The intelligent TCM assisted diagnosis and treatment model is required to have the ability to learn with few samples or even zero samples;5.In addition to the above challenges,the data collected in the real outpatient setting of the consultation and inspection needs to overcome the uncertain context in natural language and computer vision.Like various light sources,multiple shooting angles,unbalanced shooting quality and other interference factors that may affect the performance of the system,it puts forward higher requirements on the robustness and generalization ability of the proposed model.Confront the above-mentioned tough challenges,in the research process covered in this thesis,we focus on exploring neural network models for intelligent TCM applications based on attention and knowledge assistance.We consider the following benefits of attention mechanism and knowledge-assisted technology: 1.The attention mechanism is helpful for the model to more accurately capture the patient's consultation,access the key details in the input data,and improve the modeling efficiency of the model;2.Knowledge assistance can help us train a more robust model of auxiliary diagnosis and treatment,and helps to increase the interpretability of the system.Based on the above considerations,this thesis conducts innovative researches in the following parts:(1)In the online intelligent medical department classification task for Chinese patients,in response to the problems like the short text of the questions asked by the patients,the noise,and the specific strong field of key morpheme features,etc.,the framework of "attentional sequence modeling with key morpheme growth"(MG)is proposed.Extract and expand the key morphemes in the short text to enhance the influence of key morpheme features in the recurrent neural network,thereby improve the model's ability to capture key domain-guide features.(2)In the research about relationship mining of TCM prescription and tongue image,a convolutional neural network with the input of tongue image is proposed to build a TCM prescription automatically construction model.To better model the diagnosis and treatment of traditional Chinese medicine experts,the mechanism of implicit therapy auxiliary topic(AUX?LDA)is proposed to model the main subject knowledge of Chinese medical law,using a multi-tasking architecture to simultaneously learn the retrieval process of key herbs and the implicit construction process of Chinese therapy.The system achieves more realistic automation Chinese medicine prescriptions generation effects.(3)To further improve the modeling efficiency of deep convolutional neural networks,an inner-imaging channel-wise attention structure for convolutional neural networks(In I-Net)is proposed.Where the concentrated signals of CNN feature map are rearranged to form a pseudo-inner-imaging map,we use it to organize the complementary relationship of the convolution channels.Then,the proposed model uses the multi-size filters to model the convolution on the pseudo-intra-imaging map.The grouping relationship between the channels enhances the diversity,complementarity and the completeness of the overall convolution modeling.(4)On the basis of the previous part,we utilize a convolutional network based on the inner-imaging mechanism for the prediction of the patient's diseased location based on the tongue image.This is the real task of TCM-assisted diagnosis and treatment.To model the detailed pathological features scattered on different regions of the tongue image,a Fully-Channel Regional Attention Network is proposed.We also propose a stochastic regional pooling(SRP)technique in addition to the inner-imaging channel-wise attention structure.The feature map samples the signals of multiple local regions and serves as the input of the channel attention mechanism,which helps the model to automatically shield the noise signal at the edge of the image while strengthening the weight of the detailed pathological features on the tongue image.(5)In the final stage,this thesis explores the mechanism that the structured knowledge graph assists the deep neural network in completing zero-shot learning.First,a knowledge graph is established based on the co-occurrence relationship between semantic attributes and then a graph-based visual-semantic entanglement network(GVSE)is proposed,which leverage graph neural networks to mine the implicit features in visual features,simultaneously fully interact with the convolutional visual network,and finally obtain zero-shot learning features with excellent semantic expression capabilities,helping the model overcome the domain drift problem in zero-shot learning to some extent.This part of the research has laid a good foundation for the subsequent solution to the zero training sample situation in the intelligent diagnosis and treatment of traditional Chinese medicine.