Machine Reading Comprehension Based On Multi-granularity And Attention Mechanism

With the revival of in-depth learning,artificial intelligence has once again set off a climax of research,and for natural language processing,natural language has always been known as the crown in the field of artificial intelligence.Among them,reasoning ability is a key factor in the realization of real artificial intelligence.In order to enable the machine to obtain stronger reasoning ability,some scholars have proposed to use the machine reading comprehension task to enable the machine to obtain deep reasoning ability.And with the rapid development of various types of data sets and models related to reading comprehension,machine reading comprehension has rapidly become one of the most popular research directions in the field of nlp.Baidu put forward the DuReader2.0 dataset in 2018.Compared with other reading comprehension task datasets,DuReader2.0 is very different.Baidu has rich data sources,complex problem types,and long text data.And each question corresponds to multiple articles and multiple artificial answers.Aiming at the machine reading comprehension task of multi-document and multi-answer such as DuReader2.0,this paper mainly studies the effective information extraction of long text and the effective fusion of article and question information from three aspects.And study how to obtain more abundant text information.The main contents and achievements of this paper are as follows:The main contents of this paper are as follows:(1)the BiDAF model with good performance on single document problem span extraction task(the benchmark model given by Baidu official)is studied and reproduced,and the performance of BiDAF is taken as the benchmark performance of this task.The BiDAF model is used as the benchmark model for our work improvement.(2)for the machine reading comprehension task of multi-document and multi-answer such as DuReader2.0,this paper first introduces a new paragraph extraction strategy to extract the text that may contain answer fragments.First,calculate the BLEU-4 scores of the questions and paragraphs of the document,sort them according to the scores,select the four paragraphs with the highest scores,and the first sentence in the subsequent paragraphs,and splice them together.Intercepts fragments that do not exceedthe maximum model input range as the input text of our model.For multi-reference answers,we extract the fragments with the highest F1 value of artificial answers as the predictive answers for our training.(3)at the same time,we propose a new model,which has six layers,one is the paragraph extraction layer,and the calculation method is as follows.The second is the word embedding layer,which uses the GloVe word vector pre-trained in the DuReader2.0 corpus as the word embedding of the model.The third is the coding layer,which uses BiLSTM to code the articles and questions in order to obtain the preliminary representation of the articles and questions.The fourth is the interaction layer,which uses two-way attention to integrate the information of the article and the problem.It calculates the similarity matrix and uses softmax to obtain the attention weight in order to obtain the article representation of the problem perception and the problem representation of the article perception.Add a layer of self-attention to further integrate information about articles and questions.Fifth,the convolution layer,using convolution+ highway+ deconvolution layer to obtain more abundant article information.The sixth is the output layer,which uses the softmax function to obtain the beginning and end position of the predicted answer.In the processing of training,the author introduces a loss function for multiple answers,that is,the maximum likelihood loss function is summed and averaged to improve the performance of the model.The experiment in this paper is based on the DuReader2.0 data set,and the experimental ablation analysis and comparison with the benchmark model are given in the development set.The experimental results show that the model proposed in this paper has a certain effect on the scores of ROUGE-L and BELU-4 compared with the benchmark model BiDAF given by Baidu officials.