Research On Short Text Similarity Algorithm Based On BiLSTM And Attention Mechanism

With the development of big data technology and the Internet,information has grown rapidly.As the main carrier of information,text has delivered a variety of content,so short text similarity is studied.Short text similarity calculation refers to the calculation of the similarity value of two texts by a certain method or model.It has important value in science and engineering fields such as information retrieval,intelligent recommendation,and question and answer classification.At present,the research of short text similarity can be divided into unsupervised similarity calculation based on distance and supervised similarity calculation based on deep learning This paper studies these two models separately,and the main work is as follows.This paper first studies the traditional distance-based methods of calculating text similarity,such as Jaccard coefficient,cosine similarity,edit distance.Traditional text similarity calculation methods mostly calculate the similarity of sentence pairs from the character shape and word order.Although the similarity value can be calculated quickly,the classification effect often depends on the artificially set similarity threshold,which is not ideal.Since the similarity threshold has an important influence on the calculation of text similarity,this paper designs a method to calculate the similarity threshold to make the selection of the threshold more accurate.On this basis,a hybrid text similarity calculation model combining Jaccard with semantic information is proposed to improve the classification accuracy of traditional models.Experiments are performed on three data sets,and the method is compared with the traditional text similarity method.Experimental results show that semantic-based word embedding can effectively improve the accuracy of traditional distance-based text similarity calculation models.This paper also studies the supervised similarity calculation model based on deep learning,and proposes a short text similarity model based on BiLSTM and attention mechanism.It uses the twin recurrent neural network to study the short text similarity problem,and constructs two identical neural network model in the vertical direction.The word2 vec word embedding layer model is used to describe the characteristics of short text,which generates word embedding according to the window size.After the embedding layer,a two-way LSTM is used to extract deeper features of the sentence.LSTM has a memory function and is very suitable for processing time series data.After long and short-term memory network coding,the key part of the sentence cannot be reflected.On this basis,the recently popular attention mechanism is added,and finally the semantic similarity of sentence pairs is calculated in the fully connected layer,and the two-category cross entropy,accuracy and variance are used as evaluation criteria.Compared with other models on the MSRP data set and Quora data set,experiments have shown that BiLSTM that extracts information from two directions is better than one-way LSTM,and the attention mechanism can indeed improve the model and achieve better experimental results..