Query Expansion Based On Supervised Learning

With the rapid development of Internet technology, information in internet in the present exponential growth, and the rising popularity of the internet gradually changing people’s main way to obtain information. In the face of huge amounts of data in internet, how to obtain the information user wanted rapidly and accurately has become an important problem need to solve. Information retrieval technology provides effective help for users to obtain information from internet, but the problem of term mismatch can lower the accuracy of the system because some irrelevant documents will be returned to user under this situation. As a key component of query optimization, query expansion plays an important role in improving the performance of information retrieval systems. According to the expansion of query, the retrieval system can understand the user query intention better, and then improve the effect of retrieval precision.In this paper, on the basis of comprehensive research on query expansion method, a novel query expansion method based on supervised learning is proposed. The common research methods in the current query expansion field are mainly based on the method of pseudo relevance feedback, but the selected expansion terms will also include some irrelevant ones. This method employ supervised learning SVM classifier to re-select the candidate expansion terms so as to further filter the irrelevant ones.The query expansion method this paper proposed filtering terms twice to determine the expansion terms. It combines retrieval model to retrieve the original query and extracts keywords from the related documents, selects candidate terms by distributions characteristics and then determines the expansion terms by supervised training classifier. Compared to traditional expansion method, various statistical characteristics are taken into consideration so it can judge the performance of the expansion words more accurately by using the supervised classification model trained by reliable datasets. Finally, combine the final expansion terms with original query into a new query and make a new research. Experimental results on TREC datasets show that the proposed query expansion method can efficiently improve the precision and recall of user queries.