Research On Website Public Opinion Analysis Platform Based On Click Stream Data Mining

In recent years,with the blowout development of the streaming media and live broadcast industry,the amount of data on the Internet has reached an unprecedented magnitude.With the continuous breakthrough of the number of netizens in my country,people's daily life,work and study have become more and more inseparable from the support of the Internet.As an important manifestation of the Internet environment,Internet public opinion has an important effect on social stability and people's physical and mental health.In the context of the explosive growth of data volume,the government has ushered in new difficulties and challenges in the supervision of public opinion,government departments are constantly trying to use new technologies to monitor the public opinion.At the same time,the development of big data processing technology and artificial intelligence algorithms provides powerful technical methods for public opinion monitoring.Relying on big data processing technology and artificial intelligence algorithms,the establishment of an effective public opinion analysis platform has become an important way to fulfil the supervision of public opinion.1)Aiming at the problem of clustering and abnormal data in website clickstream data,a clickstream data clustering algorithm SK-II based on dichotomous K-means algorithm and SOM algorithm is proposed.The algorithm comprehensively considers the high efficiency of the dichotomous K-means algorithm and the high accuracy of the SOM algorithm,and avoids the problem that the K-means algorithm is sensitive to the initial clustering center.The algorithm can improve the accuracy of clickstream data clustering,under the framework of big data processing,it can significantly improve the efficiency of data processing.By setting the abnormal data threshold,the algorithm can also detect abnormal data.In addition,the scenebased sentiment dictionary established by clustering can provide a data source for the topic relevance model in public opinion analysis.2)Aiming at the problems of single data source and one-sided analysis in current public opinion analysis,a website public opinion analysis model based on opinion retrieval that combines clickstream data and text data is proposed.The model balances the emotional factors of opinion score and topic relevance,and realizes it through the sentiment classification of LSTM and the construction of a scene-based sentiment dictionary,and constructs a short-text website public opinion analysis platform based on the opinion retrieval results.The model increases the data dimension of public opinion analysis,and balances the two opinion retrieval factors of opinion score and topic relevance,and improve the accuracy of website public opinion analysis.3)In addition,an open source cloud computing platform is designed and deployed in response to the elastic requirements of application scenarios for computing,storage,and network resources;considering the diversity of website data,build an offline and online big data storage,processing and public opinion analysis platform based on text data and log data;in response to the analysis platform's requirements for big data computing performance and computing efficiency,a fully distributed computing platform is designed and deployed.The analysis of public opinion in the era of big data has entered a new stage of development.Public opinion analysis methods based on big data processing technology and artificial intelligence algorithms can dig deeper into the status quo of implicit public opinion,it plays an important role in improving national governance capabilities and governance levels and tapping the market potential of enterprises.