Research On Automatic Question Answering Technology Based On Transformer

With the advent of the era of big data,the rapid development of computers and mobile networks.We have ushered in the 5G network with hundreds of millions of information stored on the Internet every day.People have more and more ways to obtain information.Abundant information resources have built a large knowledge base on the Internet to satisfy people's desire for unknown information.However,the huge amount of information also makes it difficult for people to quickly obtain valuable information.How to quickly filter valuable information is also a problem that needs to be solved urgently.The automatic question answering system can automatically mine the semantic information of the question according to the question asked by the user,and can quickly match the correct answer to the question and feed it back to the user,which can solve the problem that people are difficult to obtain information quickly and efficiently.This topic studies the answer selection technology in the automatic question answering system.The answer selection task is defined as follows: Given a question and a candidate answer set,we model the answer to the question and train the deep learning model to let the model automatically screen out the correct candidates for the question answer.The key to this task is how to model the answers to questions and answers to get the similar relationship between the question and the answer.At present,the mainstream method is to use Bi LSTM as the core of deep neural network modeling based on the attention mechanism.However,the accuracy of this type of deep learning modeling method is not high,because the context-related semantic features in the text cannot be obtained and the amount of semantic information obtained is limited.In recent years,the pre-training model based on Transformer has performed well in various natural language tasks with its huge semantic expression ability.Therefore,this paper adopts a pre-training model to study the answer selection task.This article proposes an answer selection model based on the fine-tuned pretraining model BERT.We explore and try the use of pre-trained models in answer selection tasks.For the classifier,this paper uses three network structures: fully connected network,DPCNN network and Bi LSTM network to explore the model differences of different classification networks for answer selection models.In addition,we explore the use of pre-training models in answer selection tasks in different dimensions.For the training method of the pre-training model,we explored the second fine-tuning training method of transfer learning based on the Ro BERTa model.Regarding the impact of the amount of data on the performance of the pre-training model,we propose a data expansion strategy based on information retrieval.We test our model on three data sets,and the experimental results show that our proposed answer selection model is very effective.Our optimal model has achieved 98.2% and94.9% evaluation indicators MRR and MAP on the Trec QA dataset.The evaluation indicators MRR and MAP on the Wiki QA dataset are 93.5% and 92.0%,respectively.The rankings in both data sets are in the forefront.