Research On Entity Relation Extraction Based On Cross-lingual Transfer Learning

With the development of the information technology,under the support of massive data and computing resources,the artificial intelligence revolution triggered by deep learning has swept the world.To make computers understand natural language and create value in actual scenarios,it is necessary to provide a large number of prior knowledge,which is mainly expressed as structured data.However,compared with the rapidly increasing amount of knowledge in the real world,the generation of structured data is still not catching up.To solve this problem,researchers use entity relation extraction technology to automatically extract the knowledge contained in the raw text.However,in non-English environments,due to the lack of labeled data and poor text representation,there is a gap between the performance of non-English models and English models.In order to eliminate the gap between languages,researchers often use heuristic methods such as parameter transfer and annotation projection to extract cross-language information.These methods ignore the differences between languages and have many limitations.To solve the performance gap between different language entity relation extraction models and the low efficiency of information transfer between languages,this thesis explores the efficient multilingual information extraction technology.The main achievements are:1.Back attention knowledge transfer for named entity recognition:This method reversely uses the attention weight of the translation model for information transfer,and uses the features extracted by the pre-trained high-resource language named entity recognition model for low-resource language named entity recognition.The alignment information of the attention weight of the translation model is more accurate than the artificial annotation,and the symmetry of the alignment information can be used to transfer the task-specific features obtained from the pre-trained model to enrich the semantic representation of the low-resource corpus.Experimental results on different languages standard datasets show that this method can effectively improve the performance in low-resource languages,especially on small datasets.2.Neural relation extraction with cross-lingual piecewise convolutional neural network:This method maps multiple languages into word vectors in the same space through a multilingual language model,then extracts the features of each language through convolution,and then obtains the bilingual features using piecewise max pooling.Finally,the bilingual features are used for relation extraction.This method effectively uses the complementarity of semantics and the consistency of knowledge between different languages.Experiments on the manual annotation dataset and the distant supervision dataset show that the model can effectively integrate bilingual features.In summary,this thesis studies the cross-language transfer learning and joint learning.Finally,I hope this article can provide some reference and help for cross-language information extraction research.