Self-attention Mechanism Based Answer Selection In Question Answering System

Question Answering(QA)system is an essential research topic in the field of Natural Language Processing(NLP).With the vigorous development of Artificial Intelligence(AI)technology,the research on QA system has also made breakthroughs.As an advanced form of information service,compared with the currently well-accepted Information Retrieval(IR)system,QA system not only supports natural language interaction,but also returns answer sentences directly,rather than related web documents.Therefore,it can better understand the user's real intent of queries,and provide users with more accurate information services while efficiently meeting the user's information needs.Answer selection,as a key step in QA system,is attracting more and more attention of researchers.Traditional answer selection methods mainly make use of linguistic approaches,focusing on develop linguistic tools to convert question and answer on the grammatical structure,which are limited on performance and practicality.In recent years,in the field of Natural Language Processing,various models based on Deep Learning have achieved great success.Meanwhile,these models have also been well adopted in the research of answer selection.However,for the existing answer selection models,the traditional neural networks,are not a good solution to solve the long-term dependency problem when modelling the context of long text due to the limitations of their internal operation mechanism,such as Convolutional Neural Networks(CNN),Recursive Neural Network(RNN),etc.Thus,it's difficult to obtain the global information of text.Recently,the Transformer neural network proposed by Google team can well extract the global information of long text with only self-attention mechanism.Therefore,this paper propose to research on answer selection methods based on self-attention mechanism,which aims to address the problem of long-term dependency that is not well solved by existing neural network and obtain the global information in question or answer text.Specifically,this paper proposes the following two answer selection methods based on self-attention mechanism:(1)An answer selection method based on the improved Transformer neural network, which aims to model the context of the global information and sequence features of the question and answer sentence as much as possible.Firstly,a multi-head self-attention mechanism is deployed in a hierarchical structure,and then deploy a BiLSTM component to form the feature extractor of the method.In addition,this paper also adopts three pooling strategies to map the representation matrix of input sentence into a sentence vector in the relationship matching layer,and correspond-ingly forming three answer selection models based on improved Transformer neural network.(2)An answer selection model based on length-adaptive neural network,namely QA-LaNN model.The model employs both the feature extractor based on BiLSTM neural network and the feature extractor based on Transformer neural network to extract the global interaction features of the input sentences,to obtain high quality sentence vector representation.In particular,the QA-LaNN model can automati-cally select the corresponding feature extractor according to the length of the input sentence,and more specifically solve the long-term dependence problem that is difficult to solve by the existing methods in modelling the context of long sentences.A series of comprehensive experiments are conducted in this paper to evaluate the proposed models on a publicly available answer selection dataset.The experimental results show that the proposed answer selection model based on improved Transformer neural network and answer selection model based on length-adaptive neural network can exceed multiple competitive baseline models in standard evaluation metrics.At the same time,the study also found that when the feature extractor based on BiLSTM neural network and the feature extractor based on Transformer neural network are applied to short questions and long answers respectively,the performance of answer selection can be significantly improved.