Research And Implementation Of Conversational QA System Based On Community-Question-Answering

With the rise of the question answering community,such as Yahoo! Answer,Baidu Knows and Zhihu,a new form of question-answer data has accumulated a lot in these websites.The data,not only has a huge number and a wide range of topic,but still are in rapid growth.How to exploit and take advantage of these question-anwering pairs,has become an urge demand in academy and industry.So,we develop a conversational QA system based on community question answering,in which we do some exploration work as follows:1.We develop a web crawler for download the webpages in Baidu Knows and Sougou Wenwen,from which we can extract question,answer,question category,answer quality and so on.Then,we create full-text index for the question-answer pairs.In retrival phase,we expanded the keywords in the query with synomyn dictionary.2.We propose a new and method for question sematic similarity based on paragraph vector and expaned keywords overlap ratio.We compare the new method with the methods taking the average of each word's distributed representation and paragraph vector as the composite vector.We can find an obvious improve in the experiments.3.Dialogue management is incorporated in our QA systems.We design and implement a simple dialogue management algorithm,which has the capability to handle the ellipsis,anaphoric references and ambiguity,making sure users get the appropriate answer quickly in a more nature way.Our conversational QA systems on question-answer pairs generated by community users,are convenient to build,especially for open domain QA systems.Through incorporating dialogue function,conversational question answering are efficient in answering task and friendly in interaction.