Mutual Information Based Semantic Query Expansion For Information Retrieval

While in the process of using information retrieval system, the general users usually cannot implement a completed and normative query, In that case, only a little available information is retrieved. In order to solve the problem, The technology of query expansion emerges, as the times require, but it isn't really matured until today. The root cause is the expansion of the past was simply using co-occurrence information of words which come from query and text mechanically, or using some resources to expand query based on the isolated keywords of the query, which couldn't solve problems radically. Only according to the results of the analysis which are from the semantic level can solve the fundamental problems.After researching and analyzing the existing query expansion methods, in this thesis combining previous research results to propose a new semantic query expansion algorithm based on mutual information retrieval. This method combines two extended way: the expansion based on statistical information and knowledge of linguistics based semantic expansion, and the extended words coming from both retrieval documents and semantic dictionaries. Combining with the values of semantic similarity in semantic dictionaries and the mutual information values of expanded words and query words, as well as evaluating expanded words a new weight, setting threshold according to the size of the weights of the expanded words to select the related words, Finally, it obtains the new query sets.Due to the general users only care about the former a several documents of the retrieval information, so improving the precision of the former documents has certain practical value. Then a re-ranked module based on the document reconstruction was added in this thesis.Finally, I implemented the algorithm. The contrast experimental results show that the retrieval performance MAP(Mean Average Precision) and the precision of the top 20 of the retrieval documents enhanced obviously by using of the query expansion proposed by the subject. In most cases, this algorithm improves the search quality, and has some practical value.