Answer Selection Based On BERT-LSTM Algorithm

Question answering system(QA)belongs to the category of information retrieval.It mainly returns the answers to questions or other related information by retrieving existing information.Question answering system can improve the quality of answering questions by understanding the user's questions.At present,question answering system is widely used in intelligent voice interaction,online customer service,knowledge acquisition,chat robot and many other tasks.Answer selection is an important part of a retrieval-based question answering system.With the successful application of deep learning technology in the field of natural language processing,such as machine translation,text summarization,question answering systems,etc.The answer selection algorithm based on deep learning has also become a current research hotspot.In the existing answer selection algorithm based on deep learning,Word2vec or Glove word embedding cannot solve the problem of polysemy,which directly affects the subsequent semantic representation of sentences;RNN and CNN have certain limitations in extracting text features.RNN cannot calculate in parallel due to its own structure,CNN learns local features of text.Therefore,in view of the shortcomings of existing answer selection algorithms in word embedding representation and text global information modeling,this paper proposes Transformer-LSTM and BERT-LSTM answer selection algorithms,and experiments on two benchmark data sets,which have achieved competitive results.The specific work of this article is as follows:The Transformer structure is easy to learn the global information of the text due to the internal Self-Attention mechanism.Based on Transformer and BiLSTM structures,this paper proposes TLAS(Transformer-LSTM)answer selection model.It not only uses Transformer to learn global information,but also uses BiLSTM to achieve semantic information fusion.Based on this,attention mechanism is introduced in the TLAS model,and proposes an interactive answer selection model ITLAS.The method has achieved excellent results on the public data set.The BERT pre-training model learns general linguistic knowledge during the pre-training process,and extracts the word embedding which contains contextual information.This embedding has better expressive power than traditional Word2vce and Glove.Therefore,this paper proposes the BERT-LSTM model,it can be fine tuned based on the BERT pre-training model,and then integrates semantic information with the BiLSTM layer.Finally,we evaluate our model on the popular answer selection data set WikiQA and InsuranceQA.Experiment result concludes that our improving algorithm proposal has better performance than several competitive benchmark methods.