Research On Machine Reading Comprehension Method Based On Transfer Learning

Machine reading comprehension is one of the most difficult tasks in the field of natural language processing and artificial intelligence,and has attracted great attention from academia and industry.In recent years,researchers have successively proposed multiple high-quality large-scale machine reading comprehension datasets,coupled with the rise of pre-trained language models,which have greatly promoted the progress of machine reading comprehension tasks.However,the lack of large-scale supervised training data is still one of the important challenges in this field.In response to this challenge,based on unsupervised domain adaptive methods at home and abroad,this paper focuses on how to improve the transferability of the extractive question answering task model in machine reading comprehension,and carries out the following work:(1)In response to the existing problem of lack of training data,this paper integrates existing methods and proposes an unsupervised domain adaptive framework based on multi-source domains(Multi-source Question Answering,MSQA).The model utilizes multiple existing large-scale labeled datasets,and on the basis of the pre-trained model,uses a domain adaptive method to simultaneously narrow the feature distribution of multiple source domain and target domain datasets,thereby improving the model’s Transfer performance on target domain datasets.Experiments show that the model has better generalization ability and task performance than similar methods.(2)The extractive question answering task is essentially a regression-like task.Although the feature distribution of the two domains can be forcibly narrowed by using the domain adaptive method to improve the transfer performance of the model,the corresponding learned features will lose the original model.task capability,so most domain adaptation methods designed for classification tasks do not perform well in regression tasks.To solve this problem,this paper designs a Balanced Norm Conditional Domain Adversarial Question Answering(BNQA)method based on a single-source domain.information from the domain discriminator,and balance the cascaded sentence features with the norm level of the model’s predicted output.Experiments show that the model can improve its adaptive performance of the model without losing the task ability of the original model.A large number of experiments on multiple widely used English extractive question answering task datasets show that the method proposed in this paper can achieve a maximum task performance improvement of 2-3 percentage points on multi-source domain transfer and single-source domain transfer tasks.Compared with related similar methods,it has superior model transfer ability..