Font Size: a A A

The Research And Implementation Of Chinese Text Classification Technology Based On Decision Tree

Posted on:2009-02-20Degree:MasterType:Thesis
Country:ChinaCandidate:Q Y YuanFull Text:PDF
GTID:2178360308979246Subject:Computer application technology
Abstract/Summary:
With the rapid development of Internet technology, the number of data files grows sharply. There are lots of knowledge with enormous potential value in these massive Web pages. How to identify potential knowledge quickly and effectively has become an important research direction of data mining technology. Text classification is an important content of wed knowledge discovery. With the tools of text classification, users can read Web content more easily. Also, by limiting the searching range, one can find the interesting content on the Internet as soon as possible. Text classification is a basic cognitive form of information. The current methods of feature dimensions reduction, text categorization and text categorization rule extraction are still far from meeting the actual needs. This paper mainly presents the text features dimensions reduction, the classification, pruning and text categorization rule extraction with the decision tree.In this paper, the text features dimensions reduction, the classification, pruning and text categorization rule extraction with the decision tree are researched. First, a dimensions reduction method based on pattern aggregation and improved chi-square principle is presented to reduce the dimensionality and enhance the classification accuracy effectively. Then, a new DC4.5 classification algorithm based on C4.5 of decision tree is proposed. Meanwhile, the pruning method of decision tree is improved. At last, by improving several important steps of classification process, new text categorization rules are extracted based on decision tree that are understood easily and have better accuracy of categorization.In this paper, first introduce the related technologies of text classification, including the text representation model, the text features dimensions reduction and the text classification methods (KNN, SVM, Bayes, etc.). Again, emphatically introduce the text classification methods with decision tree. Also, present three improving methods aim at steps such as text features dimensions reduction, classification and pruning through which greatly reduce the constructing time of decision tree and improve the correct rate of classification. It can also solve the problem of extract rules with high dimensional attribute. The following test indicates the classification algorithm implemented according to the improved methods not only has the advantage of easily extracting understandable, but also ensure the classification accuracy, improve the classification efficiency and has good practical value.
Keywords/Search Tags:decision tree, pattern aggregation, chi-square principle, text classification, pruning method
Related items