Text Classification Based On The Extending Of Core Words

With the development of information technology, especially the popularization of Internet, the number of electric text grows rapidly. It is necessary to organize the information for faster and easier processing and searching. Text classification is often used to organize and manage large sets of text or hypertext documents, but most of the conventional algorithms can not meet the requirement of current classification task, such as high dimensionality, high volume and high readability.While previous association-rule-based text classification follows such an idea: a document is treated as a transaction and frequent itemsets are mined from the transaction database consisting of all documents. However the basic semantic unit in a document is actually a sentence. A group of words co-occurring in the same sentence are usually more meaningful than the same group of words spanning several sentences.Based on the above intuition, this paper presents a new frequent itemset based text classify method. It views a sentence rather than a document as an association transaction. It mine the maximal frequent itemset or close frequent itemset in the sentence level, and use these itemset to construct prefix trees and classify the test documents. classification speed can be improved greatly. It use a novel heuristic instead of the traditional method based on the document coverage to prune the association rules, it improve the speed of pruning rules.The effectiveness of this method has been demonstrated comparable to well-known alternatives and much better than current document-level association based methods on the Reuters and Enron corpus.