Research On Reading Comprehension Style Question And Answering Model Based On Attention Mechanism And Neural Network

Natural Language Processing requires the computer to process,understand and apply human natural language,is a real "Artificial Intelligence",and has great significance to the future development of Artificial Intelligence.Machine Reading Comprehension is one of the core tasks of Natural Language Processing,which mainly involves Deep Learning,Natural Language Processing,Information Retrieval and other fields of knowledge,so that the computer can correctly understand the semantics of natural text and help people to find the ideal answer from the massive data,and thus reduce the costs of obtaining information.With the development of relevant domain knowledge,researchers proposed a machine reading comprehension style question and answering model based on attention mechanism and neural network,which can selectively screen out important message from large-scale data information and give it higher weight,and improve the model's ability to extract text features and express the combined semantic information.It has been a certain extent meet the need of people to obtain information quickly and accurately,widely recognized and applied in the field of Natural Language Processing.However,it is still a very important challenge to improve the quality of high-level semantic vector representation of unstructured text.Most of the existing question and answering models use the Recurrent neural network to directly train the high-level text semantic vector based on the pre-trained word vector.ignoring the case where the word tokens are not in the pre-trained word vector dictionary(out of vocabulary),such as proper nouns,new words,etc;At the same time,the advantages of attention mechanism are not fully utilized,and the textual semantic information extracted by the question and answer model is not prominent,resulting in a relatively poor performance of the final model.This paper systematically expounds the relevant content of the reading comprehension style question and answering model,analyzes the text vector representation and the machine reading comprehension model in the existing model.And makes the improvement to the existing deficiency,constructs a improved reading comprehension style question and answering model.The main research contents of this paper are summarized into the following two aspects:(1)For the problem of neglect to deal with out of vocabulary,resulting in weak high-level semantics information in some models,a character coding model CHAR_CODER based on Convolutional Neural Network and HighWay network is proposed based on the DrQA model proposed by Facebook.Firstly,the network structure of the Convolution Neural Network model is improved,and the convolutional kernels of different sizes and numbers are designed to extract the character-level feature vectors from different angles.Secondly,in order to reduce the influence of gradient disappearance or explosion caused by the increase of the depth of the Convolutional Neural Network,and inputing the output into the Highway Neural Network,which makes the character-level vector representation more efficient.Then,the features such as Part-Of-Speech,Named Entity Recognize obtained by the word segmentation tool,which are combined with character-level vector representation and pre-trained GloVe representation,input into the bidirectional Long-Short Term Memory network for training,and further completing the automatic extraction of a plurality of local abstract features.Finally,the obtained high-level text semantic vector is input into the reading comprehension model,and the performance of the model is verified by experiments.(2)For the problem of some key points extracting high-level text semantic vectors from question and answer models are not prominent,a reading comprehension style question and answering model AttQA based on attention mechanism and BiLSTM is proposed.Firstly,the initial word embedding of the passage and question is obtained by the pre-trained word vectors.By means of the attention mechanism,the attention of passage to question,question to passage is calculated.And then combining with the corresponding initial word embedding,use the non-linear transformation function to get the attention vector of the passage and question;Secondly,the passage and question character-level vector obtained by the character encoding model in(1)is spliced with the corresponding attention vector,the initial word embedding and the word token features,and input into the BiLSTM network to obtain the passage and question text vector representation;Then,using the attention mechanism again and combining the special gate unit to process the text vector representation,strengthen the ability to extract semantic information of the passage to question and the question to passage.And introduces Self-Attention Mechanism to solve problem of the coverage of the original text information caused by text information which is processed through multiple Attention Mechanisms;Finally,By using bilinear function,according to the high-level semantic vectors of passages and questions obtained from the reading comprehension model,the position fragments of the answers in the passage are predicted,and the segment with the largest probability fragment is selected as the answer.The experimental results show that,the improved text coding scheme rasises the EM of DrQA by 0.15%~0.76%,and the F1 by 0.19%~0.58%.The improved machine reading comprehension model is better than the EM of the DrQA,which is increased by 0.79%~1.8%,and the F1 is increased by 0.55%~1.43%.And the EM of the final model is increased by 1.8% compared with the basic model,and the F1 is increased by 1.43%.It indicates that the character-level feature and attention mechanism can enrich the text semantic information,and thus effectively improve the performance of the reading-comprehension question and answering model.