Research And Implementation Of LDA Model Topic Evolution Analysis Method Based On Modularity

With the continuous progress and development of Internet information technology,the number of network news is also rising.Facing the huge amount of network news information,it is difficult for people to quickly and accurately locate the content they want to master.LDA topic model can effectively reduce the text dimension,and the research on topic evolution analysis is also increasing.However,it is difficult to effectively select the number of topics in LDA model,which is still insufficient in related research.Therefore,this paper focuses on complex network theory,modularity and LDA model for topic discovery,and designs a topic evolution system on this basis.The main work and innovation of this paper are as follows:(1)A topic discovery method based on modularity and LDA model(MCN-LDA model)is proposed.Based on the theory of complex network,this paper constructs the relationship between nodes and edges with the co-occurrence relationship between word entropy and keywords,thus forming the co-occurrence complex network of information topic words.Subsequently,the community in the co-occurrence complex network of information topic words is divided by the community discovery algorithm based on modularity,so as to obtain the optimal number of topics and realize the effective improvement of LDA model.Experiments show that the proposed model has better generalization ability and topic extraction performance.(2)A time slice division method based on topic evolution cycle is proposed.In this paper,the theory of topic evolution cycle is added to the time slice division.Each stage of topic evolution has its own characteristics,and the duration is also different,so it is necessary to divide the time slice according to the specific situation to study the change of characteristics in the process of topic evolution.Consequently,this paper combines the topic evolution life cycle and the number of news texts in each time period to divide the time slices and realize the effective discovery of the feature changes in the evolution process of topic relationship.(3)A method based on JS divergence and time decay is proposed.On the basis of JS divergence,this paper introduces time decay into topic relation discrimination,and judges whether there is evolution relation by harmonizing the size relation between similarity and threshold.Experimental results show that the proposed method improves the effect of topic evolution relationship recognition.To sum up,this paper studies the topic discovery,time slice division,topic evolution analysis and other aspects of news topic evolution.It proves the effectiveness of the topic discovery method on public data sets,and analyzes the topic evolution on real data sets,and provides research ideas and theoretical guidance for the research of news topic evolution.