Fast Text Categorization Research

The rapid growth of text information of the Internet brings forward new standards and challenges for text categorization with respect to accuracy and speed. So the trend requests us not only to boost the accuracy but also to accelerate the speed. In order to confront the challenge, the author conducts an extensive research on the perspectives of technologies of fast text categorization, and achieves a certain extent progress.The thesis firstly introduces general development of automated text categorization. Specially, some introduces and analyses are made to compare the performances of some typical text categorization technologies such as text representation, word segment, feature selection, text categorization algorithms and so on, laying basic theoretical and experimental supports for the research in the following chapters. Then the technology and algorithm status of fast text categorization, including multidimensional indexing, sample reduction and feature reduction are generalized, especially the theory and algorithm of feature reduction. Finally, using the distribution character as the criterion for text categorization, a fast text categorization approach and it's improved approach based on the distribution character of class (DCC) had been presented in this paper. By adjusting the similarity of a text to its class based on the distribution character, the disadvantages of the imbalance of the classes and the distribution of the samples can be overcome such that the performance of text categorization may be enhanced. The experimental results demonstrate the advantage of the proposed approach in accuracy and robustness, especially in speed.