The Desgin And Implementation Of The Micro-blog Public Opinion Monitoring System Based On Spark

With the popularization of network technology,more and more users like to express their opinions on the Internet.Micro-blog is a gathering place for people to exchange information.The micro-blog relies on forwarding relationship to spread on the network,and the prediction of weibo retweets can predict the influence of public opinion of microblog in advance.At the same time,when a micro blog is propagated on the user's attention network,if it is forwarded by some influential users,the retweets of the microblog may increase sharply.In this paper,based on the Spark computing platform and related algorithms,based on the original data of weibo,the research on the prediction of the time and time of microblog sharing and the problem of the micro-blog forwarding detonation point are studied.The main tasks are as follows:1)Research on the prediction method of micro-blog time-sharing forwarding quantity.A fusion method based on text similarity and time series model(TS-ARMA)is proposed in this paper,which is related to the relationship between the trend of micro-blog forwarding and the post publication interval.First,combining the word segmentation and text similarity algorithm,we calculate the source micro-blog similar micro-blog set.Secondly,we calculate the time series characteristics of the initial time based on the similarity micro-blog and the similar micro-blog weight value.Finally,based on ARMA model,we predict the forwarding quantity of micro-blog in different time.At the same time,this paper is based on the XGBoost algorithm to study the micro-blog time-sharing forwarding prediction,focusing on the micro-blog user fan feature and the active time of micro-blog users.The prediction of micro-blog forwarding volume is refined to the time interval after its publication.In the aspect of micro-blog's public opinion delivery timeliness,micro-blog's influence on public opinion at different times is identified in advance,so as to achieve the role of monitoring.2)Research on micro-blog forwarding explosion point analysis.In this paper,FP-Growth frequent itemsets mining algorithm of frequent users may exist in the forwarding mode based on the proposed based on frequent forwarding users in the network point method corresponding with the user of micro-blog by forwarding speed combination,to determine its possibility of forwarding the explosion of micro-blog.TS-ARMA algorithm experimental results show that,in the case of sufficient micro-blog history,the TF-IDF and time series fusion method has better prediction effect on time sharing forwarding.In terms of XGBoost algorithm prediction,the experimental results show that the maximum hit rate of 62%.By adjusting the forwarding area,we can further improve the hit rate.As for micro-blog explosion point decision,the experimental results show that users can be transmitted more frequently,combined with the frequent forwarding relationship between users and the forwarding speed of micro-blog,the three way is combined to identify users,which is likely to become micro-blog explosion point.