Font Size: a A A

Research On Clustering Algorithm For Web Document By Incorporating Distribution Information

Posted on:2009-01-03Degree:MasterType:Thesis
Country:ChinaCandidate:C H SunFull Text:PDF
GTID:2178360245975963Subject:Computer application technology
Abstract/Summary:
With the rapid development of the internet, the information resources on the Web have covered all the fields of the society, the issue of overloading information becomes more serious day by day, which boosts the development of the Web Data Mining Technique. In this paper, from the viewpoint of web document clustering, we do our research on the representation of distribution information of a document and the corresponding similarity measurement, and multi-views clustering, and kernel based multi-views learning. The main contributions of this paper are as follows:1. Propose a similarity measurement method which incorporates distribution information. Most of the existing Web Data Mining techniques are based on VSM, which only achieves some effects, and does not concern other useful information contained in the web document. In this thesis, we introduce a new similarity measurement method with the distribution information of the word contained in the document, which is an extension of the traditional similarity measurement. Experiments show that, the new similarity measurement in this thesis has better clustering performance than the traditional similarity method.2. Propose a new mult-view algorithm. In this method, different algorithms have been applied on various views, which can express the distributional features of the document in the data set more clearly. Experimental results show that the accuracy of the classification has been improved.3. Propose a kernel-based co-training clustering algorithm. The different kernel functions can induce different distances of the original samples in original space. In this thesis, plenty of tests have been performed by using Polynomial Kernel and Gaussian Kernel; the results show that after adopting the kernel methods, the multi-view algorithm of clustering have been apparently improved.
Keywords/Search Tags:Distribution information, Clustering, Web document mining, Kenerl function
Related items