Research On Intelligent Question-answering System Based On Deep Learning

In recent years,natural language processing is playing an increasingly important role in human's life.The emergence of intelligent Question-Answering has led to a large number of applications or services,such as Baidu's Xiaodu,Ali's Tmall Genie,Xiaomi's Xiaoai classmate,and Apple's Siri.However,the field of intelligent question answering is substandard yet.Improving the accuracy of the intelligent question answering system makes the machine more intelligent and answer the questions closer to or even beyond the level of humans.The research in this article is based on this purpose.Firstly,I introduced the basic model of natural language processing,from one-hot encoding,word embedding to sequence to sequence,and then I introduced the pre-trained model originated from Transformer,especially BERT which is Bidirectional Encoder Representation from Transformers,marking the natural language processing has entered a golden period.The concept and model of natural language processing has laid a solid foundation for further research on intelligent question answering.Then,based on the bidirectional attention flow of the basic model,that is BiDAF,the Bidirectional Attention Flow model.I used the dataset SQuAD 2.0 which is Stanford Question-Answering Dataset as training data,and used the values of Exact Matching and F1 as evaluation indicators.EM stands for whether the predicted answers exactly the same as the groud-truth answers.The greater the EM value,the closer the model predicts to the standard answer;the F1 parameter which based on the coincidence of the answer given by the model and the standard answer is to find a score between 0 and 1.This score is a harmonized average of the accuracy rate and recall rate.After training,the performance of the BiDAF model on the SQuAD 2.0 dataset is that the EM value is 58.60 and the F1 is 61.95.Finally,the pre-trained model ALBERT(A Lite BERT)is used for research.There are 5 models for it.The first model uses ALBERT directly plus the output layer,the second model adds a highway network on the basis of ALBERT,and the third model adds the Gated Recurrent Unit and Attention layer,the fourth model uses GRU(Gate Recurrent Unit)and highway network,the fifth model is to use ALBERT-xxlarge version.After training,in the model 1,2,3,and 4 which uses the ALBERT basic version,the first model which directly adding the output layer to model has the best effect,which is 17.68 higher than the basic BiDAF model on EM value,and the F1 value is increased by 16.73.Among the five models,the ALBERT-xxlarge version has the best effect.The EM value is increased by 24.35 than the BiDAF model,and the F1 value is increased by 23.35.The innovation of the paper is using the most powerful model nowadays,rather than BERT or XLNet.On the basis of ALBERT,a method of adding different layers for research is proposed,which improves the accuracy of SQuAD 2.0 greatly.The ALBERTxxlarge model with the most parameters has the best effect,which is 41.55% and 37.69% higher than the EM value and F1 value of the basic model,respectively.The effect is remarkable.