Reasearch On Machine Reading Comprehension Based On Attention Mechanism

The goal of natural language processing has always been to enable machines to understand text,achieve human-computer interaction,and enable machines to understand human needs,so that machines can provide people with better services.Machine reading comprehension task is a task to test whether the machine c an understand text.It is a comprehensive and complex task,including text representation,analysis,understanding,reasoning,etc.,which is quite challenging,so the level of machine reading comprehension can represent machine intelligence in some ways.Recently,with the release of large-scale reading comprehension datasets and the rapid development of deep learning technology,the machine reading comprehension task has also made new progress.The word vector obtained by training in large-scale corpus can represent the meaning of words.The recurrent neural network can effectively obtain the contextual semantic information of the text by the internal loop structure and hidden state.And the success of attention mechanism in the field of computer vision also brings inspiration to natural language processing.Therefore,this paper will study the semantic information representation method of articles and questions in reading comprehension tasks,and study how to fuse the semantic information to obtain key information by attention mechanism.Then use the key information to answer the question.The research content of this paper is summarized into the following three points:(1)Reading comprehension technology based on long short-term memory networks.First,obtain the word vector that can represent the word information,including the pre-trained word vector,the character-level word vector based on Char-CNN,and the lexical features extracted by the spaCy tool.Then combine the word information,and pass them to the deep bi-direction long short-term memory networks to obtain the information of articles and questions,and finally use a full connected network or bilinear classifier to predict the answer.(2)Reading comprehension technology based on attention mechanisms and multilayer connections.Apply multiple attention mechanisms to the reading comprehension model so that the information of the articles and questions can be fused to obtain key information and predict the answer more accurately.At the same time,in order to optimize the model training and reduce the loss of information in the network flow,we also use a multi-layer connection technology,so that information can be transmitted between layers.Finally,the accuracy of the model is further improved by model ensemble.(3)Build a reading comprehension visualization system.Based on the previous research,we build an online end-to-end reading comprehension system.The system has a probabilistic visualization interface for model predict ions,which can visually display the answer prediction process.