Sentence-embedding And Similarity Via Hybrid Bidirectional-LSTM And CNN Utilizing Weighted-pooling Attention

In the past decade,research on text understanding and information retrieval have produced prodigious attention among researchers for analyzing the sentence similarity in natural language processing.Although,conventional methodologies employed to operate similarity systems entirely depend on hand-crafted features.Recently neural networks have received considerable attention in sentence similarity measuring systems due to their efficiency in dealing with semantic composition.However,existing neural network methods are not sufficiently effective in capturing the most significant semantic information buried in an input.Moreover,the increasing number of applications of the deep neural network have diverted interest from wordlevel to larger texts,such as sentences embedding.To address this problem,a novel weighted-pooling attention layer is proposed to retain the most remarkable attention vector,order pattern and ignoring irrelevant words.It has already been established that long short-term memory and a convolution neural network have a strong ability to accumulate enriched patterns of whole sentence semantic representation.The combination of both models raises the ability of a model to extract comprehensive contextual information.First,a sentence representation is generated by employing a siamese structure based on bidirectional long short-term memory and a convolutional neural network.Subsequently,a weighted-pooling attention layer is applied to obtain an attention vector.Finally,the attention vector pair information is leveraged to calculate the score of sentence similarity.Investigations show that the proposed method outperforms the state-of-the-art approaches to datasets for two tasks,namely semantic relatedness,and Microsoft research paraphrase identification.Experiments were conducted on different parameters values for LSTM cell units including dropout probability and have proven its superior text semantic capturing ability compared to other existing attention mechanisms.In addition,the comparison with the recent state-of-the-art approaches manifests its competitive performance on all datasets and has established the efficiency of the proposed model to a recommended level.The developed model has also increased the learning capacity as well.An amalgamation of both,bidirectional long short-term memory and a convolutional neural network has resulted in a model that enhances information extracting and learning capacity.The best value of Pearson's rank correlation coefficient is 0.0259 and accuracy is enhanced up to 3%,thus obviously outperforming the existing baseline methods.The new model improves the learning capability and also boosts the similarity accuracy as well.