Research And Implementation Of Network Community Detection Based On Distributed System

In recent years,the popularization of online social networks and smart mobile devices makes more and more people regard online sociaty as an important part of life,which also leads rapid growing of the data in online social networks.As an important direction of social network research,community detection is of great significance in studying the characteristics of network structure,analyzing user relationships,exploring ways of disseminating messages and grasping the trend of public opinion.At present,there are many community detection algorithms of different genres in academia,they are optimized according to the data attributes of the graphs,the characteristics of the community structure and the algorithm itself.Massive online social network data brings new challenges to community exploration.Due to the marginal effect of single hardware configuration upgrade and the limitations of some traditional community detection algorithms in dealing with extremely large data,the distributed computing model is an excellent solution to deal with large-scale social network data.Based on the existing distributed community detection algorithm,this paper improves the traditional Louvain algorithm and proposes a distributed optimization Louvain algorithm based on label propagation partition,called LPPDLA.The algorithm is applied to the community detection prototype system.The main work of this paper is as follows:(1)This paper Analyzes the requirements of distributed community detection and proposes three improvements:First,size constrained label propagation algorithm for graph partitioning is used to ensure the stability of the input;Second,the use of VF algorithm simplify the node in graph and optimize community detection Computing efficiency;Third,the concept of virtual nodes is introduced to enhance the association between partitions,and node mobility rules across partitions is developed.(2)Combining the above three improvements,LPPDLA is proposed.On the basis of MapReduce model,the algorithm uses the size constrained label propagation algorithm to graph partition and uses the VF algorithm to simplify the partition data.Finally,the improved distributed local moving and distributed community contracting are calculated to complete the Louvain algorithm,and the community detection results is got.(3)Designing and implementing a community detection prototype system,which provides data crawling capabilities,can read the existing graph files,calculate with distributed community detection and display the detection results through the visual interface.Through the comparation and analysis of experiments,the LPPDLA proposed in this paper can effectively and accurately detect the community of large-scale map data,and the quality of the detection results reaches the same level of the original Louvain algorithm.