Research On Searching And Selecting Answer Sources In Question Answering System

Question Answering(QA)is a very popular research direction in the field of natural language processing.There are many technologies involved in the question answering system,the most important part of which is the search and selection of answer sources.The answer sources searching aims to filter the text collection that may contain the answer to the question from a large-scale data set,and generally uses the traditional unsupervised method.The answer sources selection is to select or extract the answer that can answer the user's question through the deep learning method based on the existing candidate text collection.This paper focuses on the answer sources search and selection tasks in the QA system,including the following three parts:(1)Density Priority Strategy for Answer Sources SearchExisting search methods for answer sources are usually based on co-occurrence of words,without considering the distribution of the keywords in the question.In this paper,we find that high and dense snippets containing keywords are more likely to be good candidate answers.Based on this,we propose a density priority strategy based answer sources selection method.The solution principle is to use the maximum distribution density of the question keywords in the text to measure the relevance of the question to the document.The design motivation is derived from the observation results of actual data,that is,the distribution of question keywords in the correct answer sources often reflects a dense phenomenon.Also,in this study,we also compare the performance of traditional methods and deep learning methods at different sentence granularity,and provide a reference standard for the trade-off between the two in practical applications.(2)Multi-granularity Interactive Fusion for Answer Sources SelectionAt present,a large number of deep learning methods have been successfully applied to answer sources selection tasks.Among them,the interaction-based model has achieved significant results.The core problem of interaction-based methods lies in the construction of interaction matrices(matching matrices).Existing answer sources selection methods only consider one type of linguistic features when constructing interaction matrices,namely the feature of word level or sentence semantic level.In contrast,this paper finds that the common use of multiple types of linguistic features helps to enhance local correlation representations.Therefore,this paper proposes a multi-granular interactive fusion based method.This method not only obtains interaction information between language units of the same granularity,but also obtains interaction information between language units of different granularities.Based on this,this paper fuses these different interactions,and applies convolutional neural networks to the interaction matrix according to previous work to decode the correlation between global questions and answer sources.(3)QA System Prototype ImplementationCombining the research on the answer sources search and selection methods in the previous article,we uses Chinese Wikipedia data,and through the storage and retrieval architecture ES,builds a information retrievel based QA system prototype based on the front-end Vue,Bootstrap,and back-end Tornado framework.The user enters a questions,and the system can combine the density priority strategy and multi-granular interaction fusion method proposed in this article to search and select from Chinese Wikipedia data,and finally return the sentence or paragraph most likely to contain the answer to the user's question.In this paper,from the whole to the part,according to the problems and phenomena found in the answer sources search and selection tasks of the QA system,we propose a solution and implementation strategy are designed respectively.Both of them have reached the leading edge performance in the experiments of their respective tasks.At the same time,this paper integrates the above two algorithms from local regression as a whole,and builds a information retrievel based QA system prototype based on Chinese Wikipedia data.