Research On Natural Language Semantic Representation And Reasoning Based On Neural Networks

Cognitive intelligence is designed to achieve the human-like ability of understand-ing,association,reasoning and so on.It is an important field of artificial intelligence.Natural language semantic representation and commonsense reasoning are the key tasks of cognitive intelligence research.Natural language semantic representation refers to the transformation of natural language into a form of semantic representation that ma-chine can understand,which is the basis of natural language understanding.At the present stage,the semantic representation of natural language is characterized by high-dimensional sparse or low-dimensional dense vector form based on statistical distri-butional hypothesis,using massive text and statistical modeling.How to improve the precision of semantic representation vector is still the key problem of current semantic representation research.Commonsense reasoning focuses on the use of commonsense knowledge and reasoning ability.At present,the commonsense reasoning method is still represented by Markov logical network and Bayesian network and other traditional probabilistic logical reasoning methods.These methods often have problems such as complex model structure,prior information dependency,low efficiency and poor scal-ability.This paper focuses on the research of natural language semantic representation and reasoning method based on neural networks,This paper studies the semantic represen-tation of words,the neural network model for commonsense reasoning,the automatic construction of commonsense knowledge base and natural language reasoning system,including:First,we study the semantic representation of words based on the fusion of multi-source information and neural network modeling.The semantic representation of exist-ing words is dependent on the statistical distribution of massive text,and the accuracy of semantic representation is not ideal due to text noise and ambiguity.Therefore,this paper proposes a semantic word vector construction method and a part of speech en-hanced word vector method under the supervision of lexical information,massive text and lexical semantic knowledge.Through the use of multi-source information such as semantic knowledge base and part of speech in the process of neural network training,the accuracy of word semantic representation is improved,and the performances on multiple natural language understanding tasks are improved.Secondly,the neural network modeling method for commonsense reasoning is studied.In this paper,considering the problem of sparseness and generalization in tra-ditional reasoning methods,the continuous semantic space representation is introduced into commonsense reasoning,and the neural association model is proposed.The model maps a large number of natural events into continuous semantic space,realizes the uni-fied modeling of the association between events by using the deep neural network,and finally completes the commonsense reasoning based on event association.Experimen-tal results on multiple natural language understanding and reasoning tasks show that the neural association model achieves better performance than existing models and has good knowledge transfer learning ability.Thirdly,this paper studies the automatic construction method of commonsense knowledge base based on massive text.Aiming at the problem of the scarcity of com-monsense knowledge base and the high cost of manual construction for them,this paper proposes a method of construting causal knowledge based on massive text.The method constrains the construction space of the commonsense knowledge base by defining a common vocabulary,and then conducts kernel sentence extraction and automatic anal-ysis based on the massive text,and finally get a large number of causal relationship between the phrase as a commonsense knowledge base.Based on the above methods,this paper has completed the construction of the knowledge base containing more than fifty thousand causal phrase pairs,which provides data support for the follow-up con-struction of natural language reasoning system.Finally,the natural language reasoning system for cognitive intelligence evaluation is designed and implemented.On the basis of the above research work on semantic rep-resentation,commonsense reasoning model,knowledge base construction,The natural language reasoning system for Winograd Schema Challenge(WSC)evaluation task is constructed.In this paper,a causal reasoning system based on commonsense knowledge base and neural association model is designed and implemented for the commonsense reasoning subtask,and the automatic commonsense reasoning on WSC causal subset is completed for the first time.for the coreference resolution subtask,a reasoning method based on knowledge enhancement semantic model is put forward,and the semantic word vector technique is used to integrate the commonsense knowledge into the word vector construction process.The unsupervised semantic feature extraction and reason-ing in the absence of task-related training data are realized.The system constructed by this method achieves the best performance in WSC 2016 evaluation.