The Study Of Relation Extraction And Knowledge Graph Based On Representation Learning

In the field of Natual Language Processing(NLP),textual information extraction has achieved extensive attentions,which plays an important role in the advance of natural language understanding of machine.Textual information extraction system devotes to convert unstructual text information(natual language)to structural information format(such as entity-relation triples).That is because the structural information is more friendly to machine and prone to understanding by machine.Conventional information extraction solutions are in the feature-engineering way,where features are extracted by linguistic experts or NLP tools;Then,linguistic features are encoded into vectors where each dimension has explicit meaning and is independent to each other.Based on these feature vectors,the parameters of classifiers are learned via supervised signals.However,this setting has two distinct shortages:the expensive cost of human annotation and the phenomenon of dimension disaster.These problems constrain the generalization ability and obviously increase the difficulty of model training.To alleviate these problems,the rapid development of distributed representation and deep learning makes significant contributions.The vector space of distributed representation is continuous;Moreover,distributed representation can provide more adequate information with less dimension size.Under this background,this paper focuses on the application of distributed representation and deep learning technique in the filed of information extraction from three angles.First is how to generate high-quality and task-specific word embeddings for the downstream tasks;Second is how to enhance neural relation extractors via entitiy pair information;Third is how to effectively alleviate or even relieve the dependency on human-annotated dataset.The following chapters detailedly introduce the proposed strategies corresponding to these four views:1.The improvement of traditional negative sampling strategy and the task-specific fine-tuning of generic word embedding.During the training of word embedding,the words with higher frequency yield more training times.However,through analyzing the relationship between word frequency and semantic expression,we find that the medium-frequency words(verb and adjec-tive,etc.)perform more importantly than high-frequency words(function words,etc.).Inspired by TF-IDF,we propose a more robust negative sampling strategy NEG-TFIDF based on word frequency and the number of paragraph that words appear.NEG-TFIDF allocates more reasonable sampling distribution for high-frequency words and medium-frequency words.It demonstrates that,with the help of NEG-TFIDF,word embeddings achieves better performance in the word embedding evaluation tasks and downstream NLP tasks.Generic word embedding is capable of providing natural language prior knowledge for downstream tasks.However,it still contains some limitations.For example,"good"and "bad" have great similarities in word embedding space,while this property is not rational for sentiment classification task.For this problem,we propose two task-specific word embedding retraining strategies,TS-CBOW and TS-SG.The task-specific label information is adopted to re-train the generic word embeddings in a self-supervied way.The experimental results indicate that,our task-specific word embedding can effectively improve the performance of downstream NLP tasks.2.More robust relation extraction system enhanced by entity pair information.It is proved that,deep learning technique has achieved noticeable performance in the application of relation extraction task.Through analyzing task characteristic,we find that entity pair information is crucial for relation extraction task.In this thesis,we adopt two different netural network architecture to validate this idea.First,for convoluational neural network,we propose an Entity Tag Feature(ETF).The traditional position feature exists two drawbacks,including the unbalanced training and the ambiguity definition of distance.Compared with that,ETF introduces four entity tag embeddings to encode the entity positon information and leads the network to pay attention to the entity seman-tic information.Second,for recurrent neural network,we propose an entity-pair-based attention mechanism(EAtt).Under the guidance of entity information,recurrent neural network gains better ability to extract the local trigger information for the recognition of relation type;in the meanwhile,EAtt effectively solve the drawbacks of conventional attention mechanism,including the lack of prior knowledge and the over-fitting problem.The experimental results on SemEval-2010 Task 8 show that,ETF and EAtt obviously enhance the per-formance of relation extraction.3.Robust distant supervision relation extraction via deep reinforcement learning and generative adversarial traning.The existance of noise instances is the key problem for distant supervision relation extraction.To alleviate the influence of noise instances,we propose two heuristic learning methods,DS-RL and DSGAN,to learn noise indicators without the assistance of human annota-tion.DS-RL is constructed based on deep reinforcement learning.This method first defines a performance-based reward;Based on that,noise indicator tries to increase the obtained reward via "action-environment-reward" loops.During this reinforcement learning process,the noise indicator is gradually enhanced.DSGAN is designed based on the concept of generative adversarial network.In each training step,the identification results of generator(noise indicator) are assigned the opposite labels to train the discriminator.Through this adversarial training,we get the optimal noise indicator when the discriminator receives the largest adversarial degree.The experimental results on NYT-Freebase dataset indicate that DS-RL and DSGAN are the effective strategies for the denoising of distantly-supervised dataset;Moreover,the performance of distant supervision relation extraction achieves further improvement.4.Generative adversarial zero-shot relational learning for knowledge graphs.Large-scale knowledge graphs(KGs)are shown to become more im-portant in current information systems.To expand the coverage of KGs,previous studies on knowledge graph completion need to collect adequate training instances for newly-added relations;However,it is impractical under the rapid development of internet.We considers a novel formulation,zero-shot learning,to free this cumbersome curation.For newly-added relations,we attempts to learn their semantic features from their text descriptions and hence recognizes the facts of unseen relations with no examples being seen.For this purpose,Generative Adversarial Network(GAN) is used to establish the connection between text domain and knowledge graph domain.The generator learns to generate the reasonable relation embeddings merely with noisy text descriptions;The discriminator tries to recognize these synthetic embeddings from the real rela-tion embeddings.Therefore,our generator is gradually enhanced during the adversarial training with the discriminator.Under this setting,zero-shot learning is naturally converted to a traditional supervised classification task.The proposed method consistently yields performance imporovements on NELL and Wiki dataset.