Low-resource Language Machine Reading Comprehension Based On Transfer Learning

Given a passage c and a question q,Machine Reading Comprehension(MRC)requires models to extract a consecutive span from c as the answer to q.This extractive MRC has made remarkable progress in rich-resource languages,especially in English.However,the MRC research on low-resource languages has been severely circumscribed due to the absence of sufficient training data and the difficulty of corpus labeling.Recently,as the pre-trained language models(PLMs)have become a new paradigm of natural language processing,the cross-lingual transfer method based on the multilingual PLM has achieved breakthrough progress in low-resource language MRC tasks.However,it can only roughly locate the position of the answer,and cannot obtain an accurate answer span.Existing methods study this issue in terms of answer span characteristics,task-level semantic alignment,and transfer knowledge post-processing,but ignore the problem caused by the syntactic divergence between source and target languages:the answer spans of the target language by cross-lingual transfer learning does not conform to the syntactic constraints.Syntactic information helps machines understand,but it is challenging to improve MRC in low-resource languages by using complex and diverse multilingual syntaxes.In view of the above challenges,we study the influence of syntactic knowledge acquired by multilingual pre-training models on the transfer process and propose an MRC model based on multilingual semantic decoupling representation.Furthermore,we explore the reasons why syntactic knowledge in PLM has an impact on cross-lingual MRC.Firstly,we experimentally illustrate the problem that the target language answer span and the proportion of syntactic constraint consistency decrease,and innovatively propose an MRC model based on multilingual semantic decoupling representation.Facing the challenges brought by multilingual syntactic diversity and mutual dissimilarity,a siamese network decoupling model is proposed,which disentangles the semantics and complex syntax in the multilingual pre-training representation,and reduces the negative influence from syntactic divergence by only transferring the decoupling semantics representation.It improves the effectiveness and generalizability of the MRC model in low-resource languages.Secondly,we conducte exploratory experiments based on the structural probe and found that the original syntactic information in the multilingual PLM was lost during the fine-tuning process of MRC.Since the answer span is mostly composed of one or more syntactic components,the loss of syntactic information may be one of the reasons that the answer spans of the target language violate syntactic constraints.In this paper,we introduce a new syntactic graph prediction task,which is jointly trained with the MRC task.It improves the multilingual syntactic expression and guides the MRC model to detect more accurate answer span boundaries of target languages.Finally,we design extensive comparative experiments on the proposed model.Based on two typical multilingual pre-trained models(mBERT and XLM-100)and evaluated on three public multilingual MRC datasets,the results of experiments demonstrate that our model excellent performance on 17 low-resource target languages.The ablation and analysis experiments prove the effectiveness and generalization of the decoupling model and the syntactic graph prediction task for low-resource language MRC,and the consistency between target language answer span and syntactic constraint is improved.