Research And Application Of Knowledge Extraction And Representation Learning Based On Pattern Mining And Deep Learning

Knowledge is Power.Knowledge is playing a pivotal role in human real life.It is the cornerstone of Artificial Intelligence?AI?.The process of generating knowledge is to build cognition and understand the world.Because human has the ability to acquire,form and transfer knowledge,we can make continuous progress.Currently,a machine can efficiently learn the perception ablity including visual ability and acoustic ability from massive data.However,it is hard to let the machine possess the cognitive ability,such as inference ability.The general goal of AI is to let the machine understand and freely use the knowledge.There are three phases about the knowledge:knowledge extraction,knowledge representation,and knowledge applications.The goal of knowledge extraction is to automatically extract the knowledge from large-scale free text.Currently,the knowledge extraction has the following?1?limitation.The logical representation of knowledge is easy understood for human.But it is hard for machine to understand.Thus,varities of distributed representation learning methods are proposed recently.For the concept graph embedding and knowledge graph embedding,they have the following?2?limitation.About the knowledge applications,we select two applications:similar question retrieval and relation classification.The two tasks do not efficiently leverage the knowledge information in the current learning process,which have the following?3?limitation.?1?Currently,the knowledge base is typically described as a triple?subject,predicate,object?.However,the object is not uniqueness given subject and predicate.Classical Knowledge-based question answering generally provides the same answer to questions having similar intent but with different conditions.?2?In existing knowledge graph embedding works,the concept information is just considered within a context in the concept embedding.Besides,the logical structure information exsiting in the knowledge graph,which has strong inference capability,is missing in current knowledge graph embedding works.?3?To the best of our knowledge,there is no method that can resolve five problems,such as synonymy,polysemy,word order,question length,and data sparsity,exsisting in the question retrieval task in one framework.For the relation classification task,all of the existing models consider it as an ordinary classification task,and ignore the triples.In this thesis,we propose a series of methods to address the above limitations.To summarize,the contributions of this thesis are as follows:?1?To address the 1^st limitation,we construct the conditional knowledge base using the question answer pairs or query log from a commercial search engine,and construct a dialogue model to chat with user for specifying the missing condition.Besides,a new network embedding algorithm is proposed to encode the information network,which has two distinct relation types.?2?To address the 2^nd limitation,we propose to encode concept embedding and word embedding from both context-dependent and context-independent view.Besides,we propose a distributional encoding of logical structures for knowledge graph embedding model.?3?To address the 3^rd limitation,we directly learn the question vector representation using a high-level feature embedded convolutional semantic model.A value-based convolutional attention method is used in the convolution operation.Finally,to solve the data sparsity problem,a multi-view learning method is used to train the convolutional semantic model.To encode the knowledge into the relation classification task,we propose a knowledge graph powered?KGP?method.Our proposed KGP method can be easily integrated with existing neural networks that use the pairwise ranking loss function.We propose a data augmentation method to train the relationembedding and reversed relationembedding using one sentence sample.We conduct extensive experiments for evaluating each part with the corresponding baselines.In each part,the experiments show that our proposed methods significantly inprove the performance of the task.