Deep Learning Based Knowledge Acquisition Across Domains And Languages

Harvesting structured knowledge with high-quality from plain text is essential in nat-ural language processing tasks.Knowledge in general domain has been widely used in various areas,such as semantic search and question answering systems.However,such knowledge suffers from low coverage in specific domains,due to the long-tail distribu-tion of domain data.Therefore,recent focus on knowledge extraction has shifted from the general domain to specific domains.Challenges for domain knowledge acquisition are mainly of two folds.The low-frequency of most domain knowledge results in con-text sparsity problem,which makes it difficult for traditional pattern-based approaches to achieve comparable performance as in general domain.The existence of differences between specific domains prevents knowledge transferring directly from one domain to another.Besides,knowledge sharing between different languages is also an important re-search topic.This thesis targets at cross-domain and cross-lingual knowledge harvesting via deep learning techniques.The main contributions are as follows·Exploratory relation classification in single domain Since plenty of domain relation types are incompletely defined due to the long-tail distribution,the thesis proposes a dynamic neural network model for Exploratory Relation Classification task.The model learns a neural classifier on predefined relation types,as well as discovering new relation types from unlabeled data.The proposed clustering al-gorithm,e.g.similarity sensitive Chinese restaurant process,is able to continually expand the predefined relation set.Experiments conducted on domain knowledge graph from Wikipedia show a high precision on both predefined and newly discov-ered relations· Cross-domain knowledge hierarchy acquisition In order to allow knowledge to transfer between different domains,this thesis experiments with three models on the task of concept dependency classification.Beside from a fully-shared and a partially-shared models,fine tuning a pre-trained neural language model is the new trend of transfer learning.It learns fundamental features from massive unstructured texts,which can be shared among different domains and thus helps the knowledge transfer from one domain to another· Cross-lingual terminology translation Knowledge translation from one language to another may encounter the problem that lexical distributions of different lan-guages vary a lot.To encode the inconsistency of corpora,this thesis proposes a Soft Piecewise Mapping Model from the embedding space of the source language to the target one via multiple matrices.Each matrix represents a latent topic,which gives respective weights to the training dictionary for differentiated mapping.The results of experiments on the task of bilingual lexicon induction shows a more sig-nificant improvement in specific domains than the general domain.