Large-scale Distributed Environment, Cluster Management System And Implementation

In recent years, with the advanced development of computer and network, theLarge-Scale distributed system based on Peer-to-Peer framework (P2P) is developingrapidly and has impropriated the most bandwidth of Internet. More and more PC isjoined together by wired network or wireless network. In this highly network age, it hasbecome the very important topic in the next network research to find how to usedispersed resource effectively, how to ensure service quality, how to realize the goal of"network is computer". It is the basic topic of P2P research to join various large scale,highly dynamic and extremely heterogeneity computers together to provide highlyexpansible, highly effective and high-powered interdependent service. Moreover, a greatdeal of applications based on P2P has come into being accordingly. Nowadays, theymainly focus on simple permanent memory systems, real time message systems andMulti collaboration systems. This article mainly talks about the research and realizationof the application of P2P on Job Management System.This paper is imbursed by an international cooperative project which isimplemented by stage. Now it has come through three phases. The first phase is todevelop the C/S structure of Job Management System—JC antetype; The second phaseaims to support cluster isomerous system and highly usability; The third phase aims toexplore resource sharing technology on WAN and Interact. This article talks mainlyabout the second and the third phases. At present, the project has been accepted.This article takes Job Management System(JC) which has finished the first phase asthe applied object, works over the theory and technique of Cluster Management Systemon huge distributed environment. By the deep research of Cluster and P2P techniqueand the amalgamation into Job Management System, realized the schedulabilitymultilayered cluster processed Job-Network model and the Job Management SystemPeer-JC based on P2P network Router protocol—PHB.The main work and innovation are:1. The Investigation and foundation of Cluster Management System of JobManagement System Divide the network nodes into super nodes, service nodes andcustomer nodes by the concept of hiberarchy. Join various highly dynamic andextremely heterogeneity nodes together effectively. The highly dynamic and theextremely heterogeneity character of the nodes indicates the computing capability, thememory size, the bandwidth and network resort time etc are different and the intentionof the nodes is different.2. Bring up and realize the schedulability multilayer cluster processedJob-Network model Bring up the schedulability processed Job-Network concept.Based on which bring up the three layered load balance schedulable model of"comprehensive schedulable—static schedulable—dynamic schedulable". This modelis the same with large scale, highly dynamic and extremely heterogeneity network.The first phase-comprehensive schedulable, this is schedulability on PHB (PrefixHybrid) overlay network. The Job-Network is parsed and schedulability on the secondphase. This is schedulability on peer to peer machine list, named static schedulable according to Job-Network. The author brings up a kind of static schedulable arithmeticbased on Job-Network DAG(Directed Acyclic Graph) model. The arithmetic expandsthe dynamic load balance arithmetic, works for static load balance of Job-Network. Thethird phase is to schedulable the jobs among machines, also called dynamic schedulable.3. Investigate and bring up the P2P network protocol fit for JobManagement On the nodes' three-layered structure, based on structural overlaynetwork and distributed hash table DHT, bring up the mixed P2P network Routerprotocol with prefix characteristic code matching identity, which joined variousextremely heterogeneity computing nodes together. The paper gives the detailconstructer method of PHB and demonstrates that the Router arithmetic of PHB can befinished in 3 hops.4. Investigate and build Large-Scale Job Management System—Peer-JCbased on P2P technique Build Job Management System on P2P overlay network,design and realize Peer-JC System with the character of P2P, make it fit to Large-Scaledistributed environment. Make the best of the highly dynamic and extremelyheterogeneity network fringe resource to provide more extensive service.5. Investigate and design the highly usability of Peer-JC System Based onthe analysis of recent cluster technique, bring up the real model of highly usable JobManagement System. According to this model, Job Management System runs on highlyusable machines as a kind of virtual service. Once the trouble comes, it will be shifted(FailOver). When the trouble is clear away, it can be resumed(FailBack). This canassure jobs' continuity and usability. According to this model, the paper gives the detailprint on MSCS cluster.6. Investigate and design the PHB replication strategy of Peer-JC SystemDesign the replication strategy with PHB character, bring up the replication strategywith DHT character and prove its effectivity in PHB. Compared with the otherreplication strategy in recent days, this strategy has less replicas and offers more help toimprove the performance of PHB.The research will have much active affection to cluster as well as the developmentand application of P2P technique.