Research On Open-domain Question Answering System Based On Web

Open-domain question answering system based on web combines the advantages of search engine and question answering system,thus has a huge prospect in the era of Mobile Internet.However,the system is a complex task,which needs to find the right answer in a large number of unlabeled texts on the Internet and brings a high cost to manual annotation.Hence,distantly supervision is an essential method for tagging.The noise tag problem caused by distantly supervision has seriously restricted the accuracy of the system.This thesis studies the noise reduction in the open-domain question answering system.Aiming at the shortcomings of current works in dealing with the noise of distantly supervision,a cascade denoising algorithm based on dynamic soft-label technology is proposed.On the one hand,the model re-ranks the paragraphs based on deep learning,which uses the pre-trained language model to encode the sentences.On the other hand,dynamic soft-label technology is introduced to constantly update the weight of tags in the training stage,so as to achieve the purpose of denoising at the tag level.In the experimental part,the results show that our model can achieve sustained and stable performance gains on both Chinese and English datasets compared with the benchmark models.In order to reduce cascading errors,this thesis further explores the impact of noise tag problem caused by distantly supervision on the iterative system.In this way,an iterative denoising model based on hierarchical reinforcement learning is proposed.On the one hand,the concept of package is first introduced into the open-domain question answering.On the other hand,a novel reward mechanism is designed for hierarchical reinforcement learning.In the experimental part,the results show that the model can effectively alleviate the noise caused by distantly supervision.Compared with the benchmark models,the accuracy of the proposed model improves significantly on the several datasets.Finally,this thesis uses the cascaded system proposed above to develop an open-domain question answering system based on web.This system mainly includes data construction module,answer generation module and application module.It answers users'questions in the form of dialog box.