The Research Of Adaptive Text Filtering Based On Vector Space Model

With the development of the Internet and the great improvement of computer technology, people are gradually surrounded by the information overload problem. How to effectively get useful information becomes a critical problem. Information filtering (IF) focuses on this issue and retrieves information relevant to the users' specific requirements. Especially, adaptive information filtering (AIF), which requires less information about users' need to construct the initial profile, is paid more attention by researchers. While in the process of filtering, the AIF system can adapt itself to improve the performance. The main work in this dissertation is to study how to construct a more precise initial user profile and a new approval of user profile learning.For the initial user profile constructing, this paper explores two methods based on query expansion technology. One is based on Cilin, and the other is based on Web Mining. The latter is paid more attention, which automatically acquires lexical context-specific expansions from the web, making full use of the context and high lever natural language processing technology, such as syntax analysis. This method includes two main stages: candidate expansion extraction and expansion validation, both of which mine the web using a search engine. By means of the two stages, we get very high expansion precise thus making the user profile constructed more plentiful and precise.For the user profile learning, this paper adopts an adaptive learning algorithm based on hierarchy clustering to update user profile. Through the hierarchy clustering, we obtain several document classes, each class containing an indefinite number of documents. The most relevant class will be used to update the user profile. This method effectively shield the learning process from plenty of feedback noises produced by distorted threshold and sparseness of initial information, and also imitate artificial feedback approximately to perfect the intelligence of adaptive learning mechanism.