Research And Implementation Of Graph Mining Platform Based On Parallel Iterative Framework

Graphs are the most commonly used abstract data structures in the field of computer science, and they enable a more complex and comprehensive presentation of data compared to link tables and tree structures. Many issues in real applications need to be described using a graphical structure, and the processing of graph data is required in almost all cases, such as the optimization of railway paths, prediction of disease outbreaks, the analysis of technical literature citation networks, emerging applications such as social network analysis, semantic network analysis, and the analysis of biological information networks. But with the advent of the information age, which has led to the explosive growth of information, the scale of graph-based data has increased significantly. How to efficiently handle thelarge graphs is a new challenge. This phenomenon highlights the scale of big graph data that is generated, and it is challenging to perform efficient analysis of these data.To solve the problem, we propose a new service system, which is based on mass storage HDFS, MapReduce framework and BSP framework, making massive data processing become feasible. While the introduction of OSGi framework, a dynamically scalable deployment framework, and workflow engine, making the scalability and fault tolerance greatly enhanced.Contents of this paper include:The research of OSGi-based service system:using the dynamic OSGi plug-in framework, the service system support fault-tolerance and hot-swap. With the service node, service system can integrate more than one computing framework. The hot-swap and fault-tolerance mechanisms:In the services system, every service can dynamically install and uninstall. Based on hot-swap, the services system provides multi re-runs, suspend, and continued to run.The architecture design of platform:Based on the design of services system, complete the architecture design and the detailed design of graph computing platform.The Implementation of graph computing platform:Combining architecture design and the detailed design, complete the platform development.Finally, tests on real mobile communication networks data show that our improvement is reliable and highly-efficient.