| With the rapid growth of Internet information, how to effectively cluster the massive text information has been a hot research topic in the field of text mining. Traditional text clustering algorithms usually use VSM(vector space model) for text clustering, but text vector space exist the problem of high-dimensional sparse, and with the increase of the size of the text data, vector space dimension becomes large, people need to select the text feature manually. These problems lead to the complexity of text similarity calculation and the decrease of clustering accuracy. The proposed method of similarity measurement based on compression distance provides a new idea for the research of this problem.Clustering algorithm based on compression distance has universality, field independence, parameter independence and other advantages, but applied to text clustering semantic information often has low accuracy. To solve this problem. Firstly, the dissertation propose a document feature expansion method, the method refers to the business card information of the specific terms in "Baidu Wikipedia", expand the feature of the keywords in the text, and for the keywords having multiple interpretation, do noise reduction processing, so as to improve the keywords’ s thematic contribution. Secondly, we propose a text clustering model based on feature expansion is proposed in this text, or DEF-KC for short. Compared to the classical text clustering method, eliminating the procedure of text representation, feature extraction, feature space dimension reduction and so on, an feature expansion process is added, effectively enhance the feature words expression ability, and the similarity calculation is based on the improved normalized compression distance, using the spectral clustering algorithm to cluster the data processing. Finally, the DEF-KC algorithm is implemented, and a comparative experiment is designed to verify the effectiveness and stability of the algorithm.Experimental results show that compared with the traditional text clustering algorithm based on compressed distance, the DEF-KC algorithm has better clustering accuracy and recall rate. |
