The Research Of Large-Scale Network Distributed Architecture Based On Volunteer Computing

With the development of complex network science recently, the research on the complex network has been studied deeply. Because of the guiding significance, it affects the other areas such as the value of sociology and communication subjects, it goes much farther than before. However, with the social progress and the science development, the scale of network presents exponential growth and the data size becomes more and more massive. Meanwhile, the data processes slowly, even to the extent that it can’t calculate exactly. Thus, how to process the data fast and efficiently has become a major challenge to the complex network research.Parallel computing technology makes it possible that the complex network calculate exactly.As some of the current computational framework restricts in a certain degree, such as the MapReduce can’t process well when it meets multiple iteration. While the main feature of the complex network computing is that it contains multiple iteration. The core idea of volunteer computing is using the idle resources in network to join in the distributed computing.In this paper, on account of the idea of volunteer computing, we design a new framework called DCBV, using the ICE middleware as a medium of communication. It’s for large-scale complex networks calculation. The framework mainly adopts the idea of the tasks queue, on the basis of the Master/Worker model to carry on the design, it increases the middle tier MiddleWare and then combine the idle machine within the network dynamic together to participate in the analysis and calculation of the complex network. Beyond that, we practice and assessment the DCBV framework prototype based on the improvement of the average shortest path algorithm.From the experience, we can see that the framework we designed can calculate the related parameters of complex networks efficiently and it has good fault tolerance. We can adjust the number of compute nodes at any moment and dynamically adjust the distribution of threads as well as the corresponding set of tasks. The new algorithm for the average shortest path shows well performance compared to the previous one, the improved algorithm is realized in the loose coupling computer network framework, and it obtains a good speedup ratio.