Research Of Machine Reading Comprehension Based On Mixed Attention Mechanism And Multi-Level Semantic Representation

Machine reading comprehension is an important branch of natural language processing.The purpose is to make machine understand the semantics of text and infer relevant conclusions.The main process of machine reading comprehension is to input the given articles and questions and output the answers matching the questions.Comparing with the traditional question-and-answer system,machine reading comprehension does not depend on a large knowledge base system,but seeks for the relationship between units from a small range of articles.The data sets of such tasks are derived from reading comprehension of human language proficiency tests.The exploration of such tasks is helpful for machines to better simulate human thinking,and is an important step to achieve advanced artificial intelligence.At present,the existing methods of machine reading comprehension have two shortcomings in text representation and reasoning mechanism.At present,word vectors are usually used to solve the problem of text representation.Traditional word vectors not only lack the distinction of polysemy,but also for machine reading comprehension tasks,the pre-trained word vectors often introduce the relationship of external knowledge.According to the current reasoning models of machine reading comprehension,most of them are based on single-type attention mechanism for semantic comprehension and answer reasoning,and there are still some deficiencies in the deep mining of text internal relations.We propose improvements based on two aspects of machine reading comprehension tasks: text representation and reasoning answers.Based on a multi-level semantic representation model,text representation can be adapted to the specific tasks of machine reading comprehension through character level representation,word embedding representation,Feature Engineering representation and context embedding representation.Secondly,two mixed attention mechanism models are implemented to solve this kind of task.A neural network model based on bi-directional self-attention and self-attention hybrid attention mechanism is proposed to solve the machine reading comprehension task of fragment extraction.In addition,a bi-directional self-attention and gate-attention hybrid attention mechanism is implemented to solve the machine reading comprehension task of filling in the blanks.Then the interaction between the problem and the document is obtained by bi-directional neural network coding.In order to enhance the semantic understanding ability of the model,to obtain highly matched answers to the questions.In SQuAD,CNN\Daily Mail and Children's Book Test,we carried out experimental verification on the well-known data sets of machine reading comprehension at this stage.The results show that the text representation of the model makes the words in the articles more similar,and the semantic reasoning of the model improves the text comprehension ability of the machine.The results show that the prediction results of the model far exceed the baseline of each data set.At the same time,the results in the verification set are better than those of the single models such as Self-Attention and AOA,and the characteristics of the models and data sets are analyzed.