Research On Question-Answer Pair Detection And Question Answering System Based On Online Forums

With the demands of obtaining information quickly and accurately, Question Answering Systems have received considerable attention due to their accurate and concise answers. With the development of the Internet technology, there exist massive online forum resources which contain lots of question-answer pairs. Because these question-answer pairs are posted everywhere and in real time, they cover various kinds of fields and indicate the current attention of internet users. Consequently, it is of great practical significance and research value to study the technology of detecting question-answer pair from online forums. Since questioners and answerers usually post loosely in the forum, there are some difficulties in detecting question-answer pair from online forums: 1. the form of questions in online forums are very different from that of traditional questions; 2. it is hard to handle the topic shift and overlap in threads; 3. the length of posts is short, resulting in the difficulty of extracting effective features. In order to solve the above three problems, we carry out the research on detecting question-answer pair from online forums. The main content of this thesis includes the following four parts:Firstly, this paper analyzes the effects of answer source on Question Answering System and proposes a technical solution of Question Answering System based on online forums. Meanwhile, we make an analysis of the problems and difficulties in this technical solution.Secondly, in order to detect the questions in online forums which is different from traditional questions, we study the types of questions in online forums thoroughly, and categorize the questions into declarative questions and implicit questions. We propose textual and N-gram features to detect these two types of questions, and study the effectiveness of the proposed features. Experimental results show that the mixture of textual and N-gram features performs well in extracting online forum questions.Thirdly, in order to solve the problems of topic shift and overlap , we propose an answer detecting algorithm base on thread segmentation. Besides, we explore the effect of several textual features and non-textual features in compensating post's lack of language features. Experimental results have shown that thread segmentation can filter out the irrelevant answer effectively, and the performance of detecting answers adopting thread segmentation with textual and non-textual features improves a lot compared with current approaches.Lastly, this paper designs and implements a Question Answering System based on online forums. The system can return answers to users quickly and accurately, hence it has good application effect and broad prospects.