Research On Machine Reading Comprehension Based On Deep Learning

Empowering machines to understand human language is an important goal of artificial intelligence.Machine reading comprehension is one of the representative natural language tasks towards this goal.It aims at teaching machines to read natural language text and answer corresponding questions.A certain level of language understanding ability is required to predict the answer correctly.In recent years,machine reading comprehension has received common attention from the academic and industrial worlds and has made remarkable research progress.The technical achievements have also been integrated into the realistic systems.In this process,the importance of data is becoming more and more prominent.The emergence of large-scale datasets helps machine reading comprehension to enter the era of deep learning.The continuous release of diverse datasets guarantees sustainable development.On the other hand,the structure of neural network models keeps evolving,and their performance is improving steadily,rivaling humans in many benchmarks.Despite such achievements,the research of machine reading comprehension is still facing many challenges.For example,the model performance heavily relies on large-scale data labeled by human.When the form of the question changes or facing a new domain,it takes time and manpower to label a certain scale of data again to achieve a good performance.In addition,although the improvement of the general neural network infrastructure,such as long short-term memory and self-attention mechanism,etc.,improves the text processing ability of neural models,but the current machine reading comprehension model fails to fully incorporate task characteristics in terms of model design.This thesis tries to handle the above challenges from the perspective of data and model.Investigating key techniques that solve these problems by data augmentation and model design with sufficient data or not.This thesis mainly makes the following four contributions:1.Data Augmentation with Question Generation for Machine Reading Comprehension.Determining whether the question can be answered according to the given text is a reflection of reading comprehension ability.This thesis attempts to improve the model ability in this aspect and proposes a data augmentation method by generating unanswerable questions,which makes the model can distinguish unanswerable questions better.Question generation model generates unanswerable question based on the text and the answerable question as augmentation data to help the training of machine reading comprehension models.The performances of models are improved without changing their structures.2.Data Augmentation with Resource Transformation for Machine Reading Comprehension.The automatic dataset construction method has advantages in scale and cost compared to the manual construction method.When the labeled data is insufficient,it can be used as augmented data to improve the model performance.This thesis proposes a method that automatically transforms Wikipedia into machine reading comprehension samples and builds a large-scale dataset containing nearly 300,000 samples.It can be used as augmented data for pre-training,or as a benchmark alone for research.An answer sentence extraction model is proposed to study the impact of data scale on model performance.3.Machine Reading Comprehension Based on Enhanced Candidate Options Modeling.Answering multiple-choice machine reading comprehension questions requires choosing the correct option from several candidates.The design of the related models mainly focuses on the representations and interactions of the question and text.The modeling of the candidate options is relatively simple and rough.However,candidate options can help the model to answer multiple-choice questions better,such as the clarifying the semantics of ambiguous questions or making the better choice by comparing the candidates.Based on the above analysis,this thesis proposes a machine reading comprehension based on enhanced candidate options modeling.The model collects relevant information in the text using the candidate options and the question and considers the correlations between the candidate options to choose the correct one more accurately.4.Machine Reading Comprehension Based on Sparse Subnetwork Identification.Machine reading comprehension models built on pre-trained language models obtain remarkable performance in domains with large-scale datasets.However,performance in the domains that are different from the training domain is often unsatisfactory.Using domain adaptation techniques can obtain better results.This thesis proposes to use only a small fraction of the whole parameters,which correspond to the sparse subnetworks in the source domain model,for domain adaptation.Through the preliminary analysis of the domain models,the importance of the key modules is shown to be highly correlated across different domains.Then the module importance is introduced into the subnetwork identification algorithm to find sub-networks that adapt to the target domains better.In summary,in the context of deep learning,this thesis aims at improving the reading comprehension ability of neural network models.From the perspective of data,augmentation data are constructed by generating questions and transforming resources.They are used to train machine reading comprehension models better.As to the models,the design of model structure and the choice of model parameters combine the task characteristics.These make the models can better learn task knowledge from the data.