Research On Distantly Supervised Relation Extraction Based On Deep Learning

As an important task in the field of information extraction,relation ex-traction aims to obtain the semantic relation between entities in unstructured text.The extracted relation facts are widely used in many intelligent fields such as knowledge graph construction,automatic question answering and in-formation retrieval.The traditional relation extraction methods based on rule templates and feature engineering cannot meet the needs of actively capturing new knowledge and relations contained and emerging in dynamic mass texts to a large extent.Today's breakthrough in deep learning has led the wave of artificial intelligence technology.With the rapid expansion of data resources and the significant improvement of computing power in the Internet era,deep learning has greatly promoted the development of various fields of natural lan-guage processing.The relation extraction methods based on deep learning can automatically learn semantic features and relation patterns from massive data,but the models rely heavily on large-scale training data,and manually and accu-rately labeling entity pairs and relation data in sentences requires a lot of labour and time.The distant supervision method effectively solves the problem of manually labeling a large amount of training data.By aligning the knowledge graph with unstructured text,large-scale training data can be automatically generated,so it has become a very promising research direction in relation extraction.How-ever,due to the imperfection and bias of the knowledge graph,the distantly supervised training data obtained through alignment inevitably have noisy an-notations.Therefore,how to suppress the interference of labeled noise in the training data has become an urgent problem to be solved in the task of distantly supervised relation extraction.Therefore,this thesis studies the basic theoryof distantly supervised relation extraction and the current status at home andabroad,and explores the relation extraction methods of mining the semanticsand features of the text through deep learning.The contribution and innovationof this thesis mainly include the following aspects:1.Research on influencing factors of relation extraction denoising With respect to the noisy labeling problem of distant supervision,a series of relation extraction algorithms based on deep learning and noise reduction mechanisms have been proposed.However,the existing works are mainly fo-cused on the improvement of specific algorithms.Few works have summarizedthe commonality between different methods and the possible influencing fac-tors of these denoising mechanisms.Therefore,by analyzing and summarizing the common characteristics of existing distantly supervised relation extraction noise reduction methods,three main influencing factors of relation extraction noise reduction are proposed,including the learning of prior knowledge in text,the labeling assumption of instances and confidence level of distantly supervised labels.In order to ana-lyze the influence of these factors on the performance of distantly supervised noise reduction,this study established a neural network-based relation extrac-tion model framework,which includes three modules:word denoising,sen-tence denoising and label denoising.Corresponding research schemes are used for different modules to analyze and evaluate the effect of these factors on the relation extraction from different levels of noise reduction.This study provides a very important idea for the subsequent research on noise reduction technology for distantly supervised relation extraction.2.Relation extraction technology based on label confidence self-directed learningIn the method of constructing training data through distant supervision,the relation labels obtained from imperfect knowledge graph alignment are noisy,and the instances where there are no relation between entities will account for the vast majority of the data set.However,existing methods pay more attention to sentence-level noise reduction,and use the labels obtained by the distant supervision method as the classification target of the model,while ignoring the noise of the relation labels themselves.In order to mitigate the impact of noisy labels,this thesis proposes a rela-tion extraction method based on label confidence self-directed learning,using latent labels instead of distantly supervised labels as the classification target.This study uses the self-directed learning process of label confidence to model the interaction between the distant supervision information and the relation pat-terns,thus applying the knowledge learned by the easy relation patterns to latent label learning for hard patterns.In addition,this study uses a discriminative loss function to suppress misclassification between positive and negative ex-amples.This study can effectively correct the noisy labels during training,so as to achieve distantly supervised label-level noise reduction and improve the relation extraction performance.3.Relation extraction technology based on multi-head self-attention net-workThe deep learning based relation extraction models encode the text into a semantic embedded representation through a neural network,which greatly saves the priori knowledge in the text that is very important for relation extrac-tion.Existing neural relation extraction models have achieved excellent per-formance in representation learning,but how to effectively capture the global dependencies of long sequences is still a very challenging research problem.This thesis proposes a relation extraction method based on Multi-Head Self-Attention Network(MSNet).MSNet can capture the information inside a sentence in different positions and different semantic subspaces,and obtain the long-distance dependency relation between two words without using any convolution and recurrent operations.Moreover,MSNet can execute multiple attention functions in parallel,greatly improving the computing performance of the model.In addition,this study adopts the MSNet-based label confidence learning method,and quickly integrates the easy relation patterns into the latent label prediction of the hard relation patterns through a curriculum function,and achieves the label-level noise reduction more efficiently on the basis of MSNet representation learning,further improves relation extraction performance.4.Relation extraction framework based on reinforcement learningThe rapid development of reinforcement learning in recent years has brought new ideas to the research of relation extraction.Existing relation extraction schemes based on reinforcement learning are more focused on sentence-level noise reduction,which reduces or redistributes noisy sentences to improve re-lation extraction performance,there are few works to achieve label-level noise reduction through reinforcement learning.This thesis proposes a relation extraction model for label noise reduction based on reinforcement learning,which includes two modules:policy network and extraction network.The core idea of the model is to design a policy network to obtain latent labels,that is,select a distantly supervised or model-predicted label as a latent label action through a policy function.The obtained latent la-bels can be used to supervise the training of the extraction network,and the classification performance on the trained extraction network is provided as re-ward information to the policy network to guide the optimization of the policy network.This model can achieve the double optimization of the policy network and the extraction network,thereby using reinforcement learning can dynam-ically correct the noisy label,and effectively improve the relation extraction performance.