Commonsense Knowledge Driven Natural Language Understanding And Generation

Feature representation is the foundation work of statistical machine learning.It is crucial to transform human natural language into machine readable representations in natural language understanding and natural language generation.In the field of statistical natural language processing(NLP),commonsense knowledge fills the background knowledge gaps to facilitate and enrich the semantic feature representations for texts which is one of the key factors that improve the performance of systems.This paper studies across different perspectives of commonsense knowledge including knowledge types,knowledge representations and their application scenarios.First,we mainly focus on two types of commonsense knowledge which are causal knowledge and hypernymhyponym knowledge.Human common sense consists of basic facts and connections among concepts about everyday world shared by most people.In NLP,those connections are described as typical commonsense semantic relations,such as causal relations and is-a relations.Causal relations are the fundamentals of thinking and inferencing for human-beings,while is-a relations are the basis for understanding how things in the world are organized.Thus,these two typical types of commonsense knowledge(i.e.causal knowledge and hypernymhyponym knowledge)we focused contain the most important understandings about concepts in the real world and their in-between connections.Second,we study two commonly-used feature representation mechanisms for commonsense knowledge which are commonsense knowledge base and distributed knowledge representation.Common sense knowledge base is based on network structure and organized by triples implied commonsense semantic relations.This form of representation is widely acceptable and used in various natural language processing tasks.In contrast,distributed commonsense knowledge representation models the semantic connections among entities,concepts,events and their in-between relations by transforming the semantic objects into a continuous,dense and low-dimensional vector space.It is an effective of bridging the semantic gap in NLP by leveraging the commonsense knowledge combined with statistical NLP models especially with powerful deep neural network models.Last,we study different application scenarios driven by commonsense knowledge which involve natural language understanding and generation tasks,including text reasoning,opinion mining and text generation.From the above perspectives,this paper investigates the key technologies of commonsense knowledge in the feature representation of natural language.To be more specific,our contributions include:1.Acquiring and representing commonsense knowledge based on pattern-based extraction.From the current state of research,the existing hypernym-hyponym knowledge bases such as Word Net and Probase have been widely applied to various NLP tasks.However,there is no such specialized and large-scale causal knowledge base for research community.We design explicit causal patterns and use them to extract causal pairs from a large amount of unstructured texts,then propose a causal strength ranking model,finally automatically construct the first large-scale knowledge graph with causalities named Causal Net,which serves to text reasoning tasks in NLP.2.Unsupervised learning frameworks based on graph features.Knowledge computing effectively mines useful information from unlabeled data based on knowledge base.We target at the scenario of opinion mining task in E-commerce platforms,and propose two unsupervised learning frameworks for prominent aspects mining based on the graph computing algorithms such as weight propagation.3.Incorporating commonsense knowledge into text generation based on distributed representations.The awareness of commonsense knowledge is crucial for statistical natural language processing models,which is an effective way to bridge the semantic gap.In addition,the annotated parallel data for text generation is usually scarce for the sake of expensive human efforts.We propose multiple methods of commonsense knowledge incorporation based on distributed representation learning which integrates the explicit common sense in knowledge bases and implicit common sense in pre-trained language model into natural language generation models.Those methods are applied to different scenarios in text generation in order to tackle the limitation of data scarcity and improve the performance of models.In summary,this paper studies the acquisition and representation of commonsense knowledge,and integrates the knowledge into natural language understanding and natural language generation systems.Such knowledge incorporation improves the performance on various tasks.Finally,hoping our work in this paper can help future academic researchers in this field.