Weibo Hot Topic Discovery And Analysis Based On LDA Topic Model

Weibo is one of the most important online platforms for modern people's social life.Many people in different industries use Weibo for information acquisition and communication on the Internet.The discovery and analysis of hot topics on Weibo can help to monitor public opinion and market dynamics.Weibo is essentially a self-media information communication and sharing platform with social attributes,which can generate a large amount of data in a short time.Given this,Weibo is very conducive to the formation of hot topics,this thesis is based on the consideration of the semantic level.By using the LDA theme model to vectorize the text data,a prototype system for discovering and analyzing hot topics of Weibo was designed and implemented.The main work of this thesis includes:1.The crawler of Weibo data was designed,and the method of pre-filtering was adopted.By comparing different types of accounts and selecting authenticated accounts,a large number of unreliable and low-quality data were eliminated.2.Introduce the valley distance,optimize the selection of K-values and cluster centers in the K-MEANS method,through clustering and speed growth perspectives,find and describe hot topics based on the keywords and key microblogs.3.XGBoost was introduced to refine the classification of hot topics using custom loss functions to get hot topics under a specified category,and analyzing the trends in Weibo hot topics.Based on the above work,this thesis designed and implemented a Weibo hot topic discovery and analysis system based on the LDA topic model.The system can find Weibo hot topics within a certain period of time,give key words,key Weibo,categories,and popularity of hot topics,and use the system's output to make accurate natural language descriptions of hot topics.The final experimental results show that the system implemented in this thesis can obtain good results.