A Distributed Mining Algorithm For Key Nodes In Complex Networks Based On Community Structure

The research of mining key nodes in complex networks has important significance and value of application.The structure and characteristics of complex networks is analyzed and the increasing network size is considered in this paper.The main works are as following.Firstly,a complex network model is built based on a distributed platform.The detailed design of the partition of complex network data in HDFS,the storage of intermediate result data in the algorithm and the MapReduce based multi-task computing framework are discussed.Secondly,a distributed mining algorithm based on community structure is proposed for mining key nodes in unweighted complex network.The community structure attributes of nodes and the direct neighborhood relationship of nodes is considered in the algorithm,and the community structure factor,information diffusion coefficient and information propagation dependence of the nodes are obtained.On this basis,the self importance of the node is calculated.And then the indirect neighborhood relationship of the nodes is also used to obtain the comprehensive importance of the node.The importance of the node determines the critical degree of the node in the entire network.Thirdly,a distributed key node mining algorithm is proposed based on core community for the weighted complex network.The algorithm divides the community of neighborhood nodes based on the similarity of nodes,and then merges the community of mutual merged nodes to obtain core community,and then completes the community division of the entire network.Each core node in the core community is considered as a candidated key node group.The degree attribute,weight attribute,and multi-level neighborhood relationships are considered to calculate the comprehensive importance of the core nodes.And these nodes are sorted according to the importance values in each community,and the top nodes in each community are used as the key node of the entire network.Finally,the two proposed algorithms are achieved on the distributed platform,and compared with the classic key node mining algorithm.With comparison and analysis,the advantage of performance of the algorithms is verified.