Text Semantic Similarity Research Based On Neural Network

With the development of science and technology and the advancement of artificial intelligence technology,as well as the emergence of new types of new businesses and new demands,people have gradually moved from an era of information scarcity to an era of information overload.In today's era of information explosion,people are eager to get content that is more compatible with their needs and information from the vast amount of information.In order to meet people's needs,a variety of applications have emerged,such as search engines,automatic question answering systems,document classification and clustering,document checking,document accurate pushing,etc.,and text similarity computing technology is one of the key technologies.In recent years,the research on text similarity calculation has received more and more attention.Some scholars have compiled relevant literatures and summarized some text similarity calculation methods.In general,text similarity calculation is divided into semantic similarity calculation and non-semantic similarity calculation.Traditional statistical-based text similarity calculation methods,such as Vector Space Models,Latent Dirichlet Allocation model.A common method for measuring the semantic similarity of texts is to use pre-trained word embedding using word2vec and other methods,and then use deep neural networks to extract sentence representations and their interactions.Then,train the final multi-layer from characterization and interaction.The perceptron adapts to the text semantic similarity tag.Another method is to integrate a variety of language-independent feature information such as sentence length and lexical similarity to achieve an integrated model,but also requires more real-time computing resources.In this paper,based on the shortcomings of traditional text similarity calculation and the advantages of text similarity algorithm in recent years,this paper proposes a text semantic similarity algorithm model based on neural network for the problem of sentence level.The working process of the model can be described as follows:Firstly,the text data is preprocessed,and the word vector is trained.The word embedding of the model is obtained by non-trainable word embedding,randomly initialized word embedding,randomly initialized character embedding and precise identifier four parts for connection operation;then using stacking The effective combination of the Bi-LSTM neural network and the Attention mechanism,each layer uses the tandem information of the attentional features and the hidden features of all the previous stacked layers to encode the sentences and obtain richer syntax and semantic information,and then Through the sentence matching layer,the interaction information of two sentences is obtained,and then input into the densely connected DenseNet network layer to decode,extract feature information,and finally,through the fully connected layer classification,calculate the semantic similarity between the two sentences.The highlight of this paper is that(1)the use of untrainable Glove word embedding and randomly initialized word embedding in the word embedding layer makes the embedding effect better and enhances the model effect.(2)The Bi-LSTM neural network with stack structure is adopted,and the Bi-LSTM weights of sentence 1 and sentence 2 are shared,that is,Bi-LSTM1=Bi-LSTM2,and the idea of DenseNet network is introduced.The input of the current layer includes not only The hidden state of the previous layer also includes the input of the previous layer.Through the transmission of the parameters,the model retains the previous feature information to a certain extent,and enhances the propagation of the feature information;(3)Introduced the Attention mechanism,which not only considers the information between words and words within the sentence,but also considers the interaction information between sentence 1 and sentence 2.and obtains more rich sentence semantic information;(4)In the encoding stage of Bi-LSTM,the Autoencoder was introduced.With the stacking of the network,the splicing operation caused the network parameters to increase continuously.The dimensional reduction operation was performed by Autoencoder,which maintained the stability of the network and improved the accuracy of the model;(5)In the sentence decoding stage,we introduced the DenseNet network.Due to the characteristics of the DenseNet network structure.the feature information is enhanced.the problem caused by the gradient dispersion is alleviated,and the parameter quantity is greatly reduced,so that the network is deeper and more accurate;(6)The cross entropy loss function is regularized to avoid the problem of over-fitting of the model.