| As one of the biggest sources of information,text information plays a role in spreading knowledge in the real society.However,because of its easy replication,many people for their own interests,the text written by others will be slightly modified and put on the Internet to gain benefits.This situation has resulted in the emergence of many rough text,so many text duplication websites and systems have emerged.However,in the actual use of this kind of system,there are often many problems such as large amount of data included,text data not classified or poor classification,resulting in the actual results of duplicate checking,which are not related to the content included in the duplicate checking system,but the duplicate checking system considers that the copied text.In view of the poor classification of big data text in the above problems,this paper proposes an improved unsupervised co-training(UCo-training)method based on the multi-view clustering algorithm,combined with the improved TF-IDF algorithm and LDA Algorithm.Based on the multi feature views of text vocabulary and word meaning,this method realizes the extraction of the feature matrix of text vocabulary and word meaning through the improved TF-IDF-WF algorithm and LDA Algorithm,and uses the form of unsupervised clustering to let the two views learn from each other and get higher precision clustering results.Compared with other existing methods,this method uses spectral clustering algorithm to reduce the dimension of the feature vector,retain the most effective feature value to generate the feature vector with the most abundant view information,and obtain the optimal clustering results by clustering.In order to evaluate the text clustering method proposed in this paper,the traditional LDA Algorithm,the improved TF-IDF-WF algorithm,the multi-view clustering algorithm Mv NMF and SM2 SC were selected as the experimental control group.Experiments and comparisons are carried out in a comprehensive database composed of Chinese Wiki and Fudan University corpus.According to the analysis of experimental data,the maximum NMI value of this method is 95.3%.Compared with the LDA method with the best result of traditional single view clustering method,the NMI value of this method is increased by about10%,compared with the SM2 SC method with the best result of multi-view clustering algorithm,it is improved by 6%,which proves that this method can complete the clustering task better.In addition,based on the theory proposed in this paper,a multi-view clustering text retrieval system is designed and implemented.According to the core idea of this algorithm,the system is divided into three main stages: feature vector group extraction,text clustering and text similarity calculation.Experiments show that the system can effectively and accurately complete the task of text clustering and similarity calculation.Finally,in view of the shortcomings of the algorithm in this paper,it looks forward to the feasible scheme of improving the clustering accuracy and efficiency of the algorithm,and discusses the improvement from the algorithm of feature vector extraction to the acquisition of the optimal threshold value,the use on the big data platform and the possibility of neural network algorithm as the underlying algorithm,which further enriches the future research direction of the algorithm in this paper The author and other researchers provide some research ideas. |
