Web Text Mining Research

The development of Information Technology makes Internet appear the problem of "Rich Data and Poor Information". Because of Internet's opening and heterogeneity, quickly obtaining what users need on WWW is getting more difificult. So, how to obtain the requirement quickly and efficiently is getting more and more important. As a kind of effective information retrieval technique, Web text mining receives much concern of researchers in recent years. This thesis regards this as a research focus, do the following work mainly:(1) The article has discussed the significance of Web text mining and gives the definition of Web mining systematically. The tasks of Web mining have been classified. The relation was discussed between Web mining with the traditional data mining and Web information retrieval.(2) The general workflow of Web text mining has been systematically explained. The key techniques used in Web text mining including characteristic showing, text classification, clustering were studied especially. The research subjects and applications of text mining have been introduced. In addition, we have also recommended a systematic prototype WebMiner about Web text mining.(3) This paper has introduced the basic theories of concept lattice, discussed and studied the advantages of concept lattice theory in data processing and analyzing. The shortcoming which the search system exists at present has been analyzed in depth. In text retrieval, concept lattice theory is used to the text to excavate the potential concept structure and interrelation among the concept, a text retrieval method based on concept lattice is proposed.