Design And Implementation Of Galaxie Platform Node Communication And Management Module

As distributed system becomes more and more popular, the study and application of stream computing has been subjected to the value more and more in industry. For now, traditional solution of node management and communication management have some insufficiency to support high level real-time and high flow features of the stream computing platform. For example, traditional node management solution is too complex to meet high level real-time feature. Meanwhile, ordinary socket is not efficient enough to meet high flow feature and more complicated data transmission mode.Facing the defects of the traditional node management solutions, this paper presents a node connection synchronization solution based on the concept of ’unified connection view’ to realize the node dynamic management. The solution not only provides dynamic node adding and exiting, but also enables the processing of the crashed nodes and attempts for recovering them. What’s more, the solution introduces message queue library ZeroMQ and makes organismic integration between ZeroMQ and Galaxie event mechanism which makes a good use of both advantages. Besides, the solution use grouping strategy so that Galaxie can support complicated transmission mode in system level. Finally, the solution introduces the heart beat mechanism which guarantees the availability of the nodes. Combining heart beat mechanism and node dynamic management mechanism, the availability of the node is improved.The solution described in this paper includes four parts:distributed node management submodule, ZeroMQ integration submodule, grouping strategy submodule and heart beat event management submodule. These submodules communicate to each other with interactive interfaces and coordinate with other modules.The solution mentioned in the paper is distinctive. Firstly, it provides a distributed node management to meet the high level real-time feature requirement and realize the error handling mechanism; Secondly, the solution makes an interactive integration between ZeroMQ and Galaxie, and this also makes a good use of their advantage. Meanwhile, it provides a more complicated transmission mode by working with grouping strategy. Above all, this paper presents a high technical and practical value of the solution.