Duplicate Question Detection Models Based On The Attention Mechanism

With the rapid development of Community Question Answering(CQ A)websites,there are a large number of questions in the CQA websites,of which many are duplicate questions seeking the same answers.For the newly posted question,if the CQA system could detect its duplicate questions,the answers of these questions would be the answers to the posted question,which could satisfy the needs of the questioner quickly.Therefore,how to detect the duplicate questions of the CQA systems is the key to the development of the systems.For the duplicate question detection task of CQA systems,our contributions are listed as follows:1)Many former works just took pre-trained word embeddings as model inputs,and they ignored the disadvantage that pre-trained word embeddings are not related to the context.We propose filtering word information through the sentence information,which can extract the task-specific information from the pre-trained word embeddings to enhance the information of word embeddings.2)Our porposed models are based on the Siamese network,which can take advantage of the interactive word-to-word information during the word matching process.During the sentence representation procedure,we convert the final single sentence vectors into multiple sentence vectors with the structured attention,thus capturing more sentence information.Besides,different from separately generating two sentence vectors for matching,we try to regard two sentences as a whole object and generate one integrated representation for two sentences,which is more natural and intuitive.3)During word matching and sentence representation procedures,we apply the attention mechanism to collect relevant information and structured attention to generate multiple sentence vectors,respectively.The visualization of the attention can make our models interpretable:the attention during word matching process helps us analyze the relation between words;during the sentence representation procedure,the structured attention can show us which words or phrases of the sentences or the sentence pairs affect the final model results.4)Finally,we test our models on several different CQA duplicate question datasets,and the experimental results show the effectiveness of our models.Ablation experiments on different modules verify the effectiveness of our proposed modules.Experimental results on multiple datasets show that our models can not only provide visualized interpretation to model results for us but also ensure the effectiveness of our models on the duplicate question detection tasks.