The Research On Confidence Evaluation In Question Answering And Dialogue System

Question answering is one of the most search hotpots in the text retrieval and natural language processing field. The input of question answering system is natural language based question and the output includes the exact answer with supporting documents. Answer ranking is one of the key technologies in the question answering system, and the confidence score is used to rank the answers. Given a question and an answer, the confidence of the answer is the correctness of the answer to the question. However, the current answer ranking technology is not good. So, it is important to do some research on the confidence based answer ranking.Natural language processing based dialogue system is also one of the research hotspots in the world. The input of the dialogue system is speech and the output is the exact response with corresponding instruction. State-of-art dialogue system should have the ability of identifying some phrase in the sentence in order to process interactive dialogue. However, the confidence research based on the word-level, concept-level or utterance-level can not give enough information to this issue. So the confidence research based on the phrase-level is important in the dialogue system.In this paper, the main contributions include three points:1. This paper introduced a new answer ranking method based on confidence: a new method to compute the similarity between question and answer sentence, namely, dependency relation triples matching. This method considers the information of question's interrogative part and non-interrogative part, and heuristic rules are used to expand question's relation triples to match metamorphosing answer sentences. Then, this similarity score is used as a new feature for answer ranking in open domain question answering (QA) track. Using the data from TREC conference, the experimental results show the new answer ranking method outperforms common density-based approach.2. This paper introduced a new research of parse subtree based confidence: given all the information from the parse subtree, including the results of a speech recognizer and a statistical parser, a parse subtree's confidence includes the parsing confidence of this subtree and the speech recognition confidence of the words covered by this subtree. The maximum entropy model has been used to calculate the confidence of a parse subtree. Three rich sets of features, ranging from acoustic, syntactic, to semantic categories, are investigated. The experimental results reached a low annotation error rate in a restaurant selection domain and in the SwitchBoard corpus.3. Another contribution of this paper is that we introduced a series of novel long distance and structural syntactic features. These featuers come from one level or multi-level dependency relations of parse trees. Compared with traditional word-level features, long distance and structural syntactic features can reflect the deep information of sentences. These novel features have been applied to the algorithm of computing parse subtree's confidence. The experimental results show that the long distance and structural syntactic features can significantly improve the performance of the system.