Research On Hot Topic Detection And Tracking Of Enterprise Public Opinion Based On Big Data

In the era of big data,with the development of information technology,the Internet has become an important strategic resource for the rapid development of enterprises,the hot topic directly reflects the dynamic of public opinion,and to be the key to the enterprise decision-making.Thus,the topic detection and tracking technology has attracted more and more attention and become a hot research.However,in the face of massive Internet information,how to deal with the data quickly and accurately from the information will face severe challenges.The focus of this paper is to find out the hot topics from the Internet information,and track the development trend of the topic.The main work is carried out in the following aspects:1.Topic detection technology and parallelization research.Firstly,according to the Kernel k-means algorithm of initial center points uncertainty and high time complexity,the paper proposed an improved method based on local density and single-pass;Secondly,the parallelization of the improved algorithm implementation on the Spark platform;Finally,the experimental results show that the improved algorithm has better clustering results and time complexity reduced,and parallel approach improve the ability to handle large scale modulus.2.Topic tracking technology and parallelization research.Firstly,the topic tracking algorithm for the performance test,select the better classification results of SVM algorithm as the realization technology of topic tracking;Secondly,the three-layer structure of Cascade SVM to parallel design and implementation based on Spark platform;Finally,the experimental results show that the appropriate number of partitions in parallel environment has better classification effect and high computing power.3.Design and implementation of real-time hot topic detection and tracking based on enterprise public opinion.Through the improved topic discovery algorithm and parallel processing,in the face of massive web data can quickly and accurately cluster,identify the potential hot topic.At the same time,the three layer structure of Cascade SVM parallel processing,for a large number of subsequent news reports,can quickly and accurately processing,classification,tracking the topic.