Semantic Similarity Measurement Of Short Text By Convolutional Neural Network Based On Multi-Dimensional Attention On Word Vector

With the advent of information explosion,the amount of text circulated on the Internet is rising sharply.Therefore,as a major carrier of information,text has been an important topic in the field of natural language processing.The short text similarity measurement is to quantify the degree of semantic similarity between two specific short texts.In the field of information retrieval,short text similarity measurement algorithm is helpful for preliminary classification and deduplication of articles.This paper solves the problem of textual similarity between two short texts based on Siamese Convolutional Neural Network.GloVe is used as word embedding layer to describe the features of short texts and the attention matrix is obtained by using two dimensions after the word embedding layer.In convolutional layer,convolution kernels with different granularity have been used based on different dimension of attention matrix.An overall convolution kernel is used for the feature matrix based on the attention matrix from the dimensions of word vectors while the one-dimensional convolution kernel is used for the feature matrix based on the attention matrix from the single dimension of word vectors.According to the type of attention matrix and convolution kernel,the above process has naturally classified our models into two data flows,module_A and module_B.This paper has learned from the pooling mechanism developed by people like Hua He,and adopted two different pooling mechanisms for module_A and module_B separately.In the next,the combination of two modules are compared and embedded in fully connected layer to measure the semantic textual similarity,and KL divergence is used as the loss function.This paper has conducted multiple control experiments on STS Benchmark dataset and SICK dataset.The results show that the Pearson coefficient increases approximately 1%comparing to the attention model based only on word vector and the P-value of significance test lies between 1%and 5%,which both suggest that the model has been enhanced with our method.In addition,comparing to REGMAPR with better performance,what is applied in word embedding layer in this paper is wholly based on attention mechanism,which reveals the strength of more simplicity.