Research On Recurrent Neural Networks From The Perspective Of Natural Language Semantics

Recurrent Neural Networks(RNNs)is an essential branch of many basic structures of Neural Networks.This elegant network structure is widely used in many Natural Language Processing(NLP)tasks,such as Language Modeling(LM),Text Classification,Pattern Recognition and so on.The widespread popularity of RNNs is mainly due to its two characteristics: on one hand,the RNNs converts the discretization information into a distributed representation of words,and on the other hand,the repeated use of neurons in the hidden layer of the RNNs makes it theoretically able to process sequence information of any length.Although scholars have discovered that RNNs can acquire some natural language semantic characteristics,,scholars have discovered that it can learn some natural language semantic characteristics,and try to capture the semantic relations in longer sequences through the improvement of RNNs,but in the process of evaluating and improving recurrent neural networks,it usually relies on intuitive experience and specific natural language processing tasks to find the relationship between the model and natural language semantics from the training results of the model.This method can be said to be inferred from the effect(that is,the word embedding is obtained first,and then the relationship with the semantics is searched),and it lacks perfect theoretical guidance.In order to solve the above problems,this paper establishes a relatively complete evaluation and improvement framework of RNNs based on cause and effect from the perspective of natural language semantics.The research content and innovations of this paper will focus on the following three scientific issues:1.How to effectively integrate the natural language semantic space with the Euclidean space of traditional embedded representation.This problem is the key theoretical basis for the study of RNNs from the perspective of natural language semantics.Aiming at the problems that the current embedded representations(such as various word embeddings,sentence embeddings,text embeddings,etc.)cannot represent natural language semantics well,and the implicit layer vector reference of the recurrent neural network is not clear,this paper proposes a new semantic Euclidean space from the perspective of semantics,and clearly refers to the n-dimensional vector as natural language semantics.Based on the semantic Euclidean space,several semantic measures for a natural language sequence(phrase,sentence or text composed of words,etc.)are defined,including expression meaning,semantic unsaturation and central ideas.Furthermore,the method to measure the semantic differences between sequences and between sequences and semantic points is given.The connection between semantics and n-dimensional vectors is established by the cause(semantic space and Euclidean space)and effect(semantic Euclidean space),which provides a necessary theoretical basis for a more reasonable and effective study of recurrent neural networks from the perspective of semantics.2.How to study the internal mechanism and external factors that affect the semantic expression,cognitive ability and interpretability of the RNN?This problem is the key to the deep analysis of the principle of RNN.To solve this problem,based on the semantic Euclidian space constructed in(1),the internal factors(recurrent units)and external factors(attention mechanism)that influence the performance of the RNN are respectively studied in this paper.2.1 How to measure the memory capacity of different types of recurrent units of RNNs.Although the current different types of recurrent units are designed to improve the ability of recurrent neural networks to capture long-term dependencies in text sequences and improve the memory capabilities of RNNs,they are only proved by the improvement of a certain natural language processing task evaluation metric,and it is difficult to evaluate and analyze the ability of the recurrent unit to memorize text sequences from a semantic point of view.In response to this problem,this paper summarizes and analyzes the different ways of using the hidden layer of the recurrent neural network.Relying on the semantic Euclidean space,it is the first to propose an evaluation framework for analyzing the memory capacity of the recurrent unit.Finally,the differences in semantic capabilities of different types of recurrent units were analyzed through experiments,and the boundaries of the memory capabilities of recurrent units were found.2.2 How to demonstrate the interpretability from a semantic perspective to the attention mechanism that improves the performance of recurrent neural networks.The attention mechanism contains hypotheses that can improve the interpretability of the model,but this hypothesis has not been reasonably and effectively verified.This paper proposes an analysis framework for the interpretability of the attention mechanism from the perspective of semantics for the first time,and horizontally compares how different types of recurrent units improve the semantic capabilities of recurrent neural network models under the influence of the attention mechanism.It opens up a new perspective for the study of the interpretability of the attention mechanism.3.How to better model the cognitive process of natural language text sequences.This problem is one of the key development directions to improve the recurrent neural network from a semantic perspective.In response to this problem,this paper first uses the Markov decision process to model the recurrent neural network,and finds its shortcomings in the face of word polymorphism and the recognition process of natural language text sequences.Relying on the concept of semantic embedding in the semantic Euclidean space in(1),and the in-depth understanding of the internal and external semantic interpretability of recurrent neural networks in(2),the attention mechanism that improves the ability of recurrent neural networks is introduced into it.Internally,a new type of polymorphic recurrent neural network is proposed.Polymorphic recurrent neural network is a generalized form of recurrent neural network,and it has the ability to more deeply describe the semantic cognitive process of natural language text sequences.In summary,the research results of this paper will help to have a deeper understanding of recurrent neural networks,provide fresh theories for the development of recurrent neural networks in the field of natural language processing,and have significant scientific significance and socio-economic value.