Design And Implementation Of Internet Public Opinion Analysis System Based On Sina Weibo

As the Internet is widely applied,Weibo,BBS,We Chat public account,news website and other platforms have played a crucial role in communication for the occurrence,development and evolution of Internet public opinion.In this context,proper analysis of public opinion tendency has been transferred from the traditional approach to the emerging network social platform.Public opinion research,for one thing,can comprehensively and systematically reflect the views and ideas of some netizens as well as the direction of public opinion,thus facilitate decision makers to timely identify emergencies,and make corresponding responses;For another,in case of public opinions that obey the socialist values,"fine-tuning" can be conducted to maintain society stability and solidarity.Aiming at the single analysis method and difficult data acquisition in the existing network public opinion analysis system,this paper adopts Scrapy distributed data crawling,data preprocessing,word segmentation,key feature extraction,hot spot recognition,keyword tracking,intelligent monitoring,and short text emotion analysis,develops a multifunctional network public opinion analysis system for the most popular platform Sina Weibo.The research in this paper mainly includes the following aspects:(1)Data acquisition module.By comparing and analyzing different data crawling techniques,this system selects distributed crawler technology.According to the user data of Sina Weibo in 34 provinces,the distributed crawling data acquisition module is designed and implemented across the overall framework,the tuning of the whole distributed system and the incremental data storage.Compared with other data crawling techniques,this system has customized a set of more practical network information collection tools for Sina Weibo,and realized the real-time and efficient collection of public opinion data generated by Weibo users;(2)Emotional analysis module.The classical Support Vector Machine(SVM)classification model and Bi-directional Long Short-Term Memory(BI_LSTM)neural network classification model is employed in this paper.Based on the standard data set published by Harbin Institute of Technology,it designs and implements the training,optimization,result prediction and other functions of the two classification models.Besides,the algorithms such as precision rate,recall rate,Area Under Curve(AUC)value under Receiver Operating characteristic Curve(ROC)are taken as evaluation indicators,and the classification model with better results will be applied to this system.In contrast to other emotion classification algorithms,the above two algorithms are more representative to the traditional machine learning model and deep learning model;(3)Hot topic mining module.Through the research and experiment of related papers,this system adopts a relatively concise idea: applying word frequency to seek for hot keywords,and extract the original Weibo moments according to the found keywords,then carry out hierarchical clustering,and calculate the correlation popularity,to complete the implementation of the hot module.Unlike other hot topic mining algorithms,this system effectively avoids the problems of large computation,time-consuming and less intuitive expression of hot topic;(4)Weibo intelligent monitoring module.After the requirement analysis of this module,the monitoring algorithm used in this system is the same as the hot topic mining module,the only difference lies in the input data set.Based on the set threshold,data clusters n(represents the number of clusters)are obtained,subsequently,according to the sorting results of data volume in the data cluster,the representative events are selected from each data cluster in order to realize intelligent monitoring;(5)Web service module.After analyzing the operation of selecting the development language of each module above,this system adopts open source Tornado network server framework in the background,with Python as the development language.The front-end page applies tables,bar charts,rose charts and other tools.As for functions,it designs and implements registration,login,public opinion overview,public opinion analysis and trend,event statistics,intelligent monitoring,hot events,keyword tracking and other modules.In comparison with other public opinion analysis systems,this system visualizes the calculation results to the user page in a more detailed and comprehensive way,so that achieve a better user experience.