Reaearch On Attention Based Question Answering Techniques

Question Answering(QA)is a long-standing challenge in Natural Language Processing(NLP).As one of the important tasks of evaluating machine language cognitive intelligence,it can automatically answer users' questions and meet users' needs.Benefiting from the rapid growth of Internet data,the improvement of hardware computing capacity and the development of natural language processing with deep learning,question answering technology has developed by leaps and bounds in recent years and has been widely used in daily life.In real world,questions of users are various and even more complex,so it is difficult for machine to understand exactly because of the diversity of natural language expression.In this paper,attention mechanisms and deep learning technologies are adopted to excavate the deep semantic information hidden in language expression and fully understand users' requires,realizing precise question answering function of the machine.The contributions of this paper include:1)A multi-module hierarchical question answering model based on Scaled DotProduct Attention Mechanism is established,including using character-level embeddings to process unknown words,capturing attention distributions through dotproduct for further interactive information integration between passages and questions,and using self-attention mechanism to deal with the problem of long-distance dependence providing effective decision-making information for the correct answer prediction.2)A biomedical question-answering model based on transfer learning is designed to solve the problem of the lack of domain specific training data which is caused by high cost of manually labeling.There is plenty of research on open-domain question answering(OpenQA)which has sufficient training data.Take the advantage of this,we adopt two different transfer learning technologies to alleviate the demand for training data in deep learning,and effectively train our model in the target domain(biomedical field),so as to obtain better performance with less domain specific training data.3)A multi-passage question answering technique based on multi-attention mechanism is proposed.Because of the various expressions of Chinese,especially when need the cross-passage answer verification,QA meets a big challenge.We have studied how to use multi-passage information and multiple candidate answers to enhance model learning strategies.We also design a joint training of multiple candidate answers strategy,which can improve the model performance effectively.In this paper,we use a hierarchical attention mechanism to capture the interaction between the passage and the question,and have achieved excellent performance on some public datasets.On Stanford Question Answering Dataset(SQuAD),our model gets 71.7% in EM and more than 80% in F1,which shows a better performance of our model than most leader board models.Compared with the winner of the Challenge in Large-Scale Biomedical Semantic Indexing and Question Answering(BioASQ),we gets the best performance by using transfer learning.We prove that effective data preprocessing can improve the performance of the model by more than 10%.