A RS-Based Application Research On Web Text Mining

In this paper, a RS-based model, which starts up from trained documents for web documents classification, is introduced. Taking advantage of the RS theory's features in efficient dealing with vague, indiscernible, and fuzzy information, we sets up a series of layered subsystems to reduce redundant properties from classification tables. In this way, we can not only efficiently reduce the dimension of documents but also keep the information of keywords set. As a result, rules for web documents classification are produced.With difference from the way that traditional methods perform by accumulating the frequency of keywords. We propose a new metrical function that employs the RS-based entropy by comparing function values to measure the feature of web pages. Besides, according to the unstructured and heterogenous characteristics of www, the effect of hypertext tags to keywords' weigh has been taken into account to obtain the most effective keywords for document classification.In order to gather experiment materials and meet the need of combining Information retrieval experiments with Information filter experiments, we equally developed WEBCRAWLER for collecting web documents, an engine model which adopts algorithm Hits to analyses web link architecture.Based on the collected web document materials, Relative web mining algorithms are realized in this paper, and the effectiveness of the approach is reported as well. We can prove from the result of experiment that the web text mining approach could be more efficient than other classification algorithms whatever in precision, recall rate, or in novelty of knowledge. Moreover, the technology is language independence.