Research On Adaptive Load Scheduling In Distributed File System

In the long-running process of the distributed file system, old nodes failureeliminated and new nodes are added dynamically as well as the dynamic creation anddeletion of stored data. There is a certain gap between the old and new storage nodes inthe storage capacity and performance. It will result in the load unbalancing problem. Thedistributed file system can’t maximize the performance, and even affect the availability. Itis necessary to maintain the system load equilibrium by load scheduling mechanism.The load scheduling mechanism for distributed file system is divided into two stages:replica placement scheduling and load rebalancing scheduling. For replica placementscheduling stage, existing programs are generally using the static algorithm or dynamicalgorithm with combination of various load information collected by storage nodes. Forload rebalancing stage, existing programs are generally using adjustment means such asincreasing replicas and replicas migration to correct the unbalanced state, but the processitself will bring some overhead, affecting the system services.For the distributed file system features, this paper presents an adaptive loadscheduling mechanism called SALS (Simple Adaptive Load Scheduling), including areplica placement scheduling method and a load rebalancing scheduling method. Thereplica placement method use reading and writing request queue length in storage node asjudging basis, computing heterogeneous node’s currently the longest response time tocharacterize the load state. Then it divided storage nodes into three states: light load,middle load and over load. The load rebalancing scheduling method use replicaadjustment as basis, while ensuring the Client according to the load state of the storagenode adaptively adjust its request transmission rate, reduce the too frequent retransmissiontimeout, to ensure that the storage node overload less impact on system performance.By testing the two scheduling stages of SALS and comparing analysis with otherrelated algorithms, it proved SALS load scheduling mechanism for the system’sperformance, read and write throughput substantially increased by20%, the averageresponse time increased by10%. It explains the effectiveness of the use of SALSmechanism in the distributed file system.