Research And Application Of Reading Comprehension QA Model Under The Multi-layer Attention Mechanism

Machine reading comprehension(MRC)is a key technology that enables humans to interact with machines by natural language,and it is also a core task of artificial intelligence.MRC tasks are usually presented in QA,requiring the algorithm to give correct answers by understanding the articles based on the questions.With the development of deep learning,neural MRC models continue to evolve.Hierarchical interactive attention and pre-trained models seems to have become the standard pattern for MRC algorithms,and has achieved better performance than humans on some benchmark tasks.However,further research shows that these attention-based MRC models are very fragile when faced with inference problems and interference text attacks.In addition,the instability of the deep model in training under few samples condition is also hindering it's application.The thesis studies the above key issues and proposes a new solution.The research content and contribution include the following aspects.First,fine-grained research on the learning characteristics of the question-article interactive attention and its working principles.Compare encoding layer before and after training to understand how it works.Experimenting on “The(20)QA bAbI tasks” with simplified BiDAF.By comparing the spatial relationship between the question-related text and the irrelevant text after encoding,we get the conclusion of how interactive attention works.Second,we take a comparative study of the optimization effects on contextualized representation and pre-trained models in training process.Under few learning samples condition,comparing the methods of pre-training and meta training,and explaining it.Third,based on the above research,we propose a new solution based in dynamic routing network,and show two new models DR-BiDAF & DR-BERT.Experiments on public data sets show that two new models can achieve higher accuracy and have better stability.In addition,we also performed comparison experiments on noise data containing interfering text with the new models,and the experiments show that they have better resistance to interfering text.