Research On Machine Reading Comprehension Model Based On Contextualized Word Embedding And Attention Method

A large amount of human knowledge is transmitted in the form of unstructured natural language text,so it is of great significance to enable the machine to read and understand the text.In the last few years,with the emergence of many datasets in this field and the progress of deep learning,machine reading comprehension has been widely concerned in the field of natural language processing.This paper focuses on the research of machine reading comprehension model based on contextual word embedding and attention method,and explores how to solve the problems of insufficient accuracy and slow training and reasoning speed of many baseline models.(1)Some of the classical baseline models cannot effectively combine the context information for further reasoning,which leads to the shortcomings of the model in answering some questions that need long-term association.In addition,the traditional model of word embedding cannot accurately contain the context information of the sentence,so the quality of word embedding needs to be improved.To solve these problems,this paper proposes a machine reading comprehension model based on contextual word embedding and gated self-attention.Firstly,the model introduces a gated self-attention layer to correlate long context and further reason.In the article question interaction layer,bidirectional attention flow is used to provide complementary information.In addition,pre-trained ELMo contextualized word embedding is introduced into the model.ELMo embedding is a kind of word vector containing deep context semantics,which is derived from a two-way language model based on large-scale corpus pre-training.Experiments show that the proposed model can effectively improve the accuracy of some questions that need complex context reasoning,so as to improve the overall performance of the model.(2)At present,most of the most effective classical models use recurrent neural network to encode word embedding,but the feature that recurrent neural network cannot be parallel computed makes the training and reasoning of these models slow.To solve this problem,this paper uses the convolutional encoder for word embedding coding,which improves the training and reasoning speed of the model.In order to ensure the high performance of the model,we also use pre-trained ELMo word embedding to improve the quality of word representation.By a series of experiments of control variable method,the best super parameter setting of the model is found.In addition,compared with many classical baseline models,the experimental results show that the proposed model can significantly improve the training and reasoning speed of the model while maintaining high accuracy,which makes the model more applicable and researchable.