Research On Load-balancing-based Synchronized Task Allocation For FastDFS New Storage Nodes

FastDFS Distributed File System is the mainstream medium and small file storage architecture in today's network environment.It has the characteristics of lightweight,high performance and high scalability,and is widely used by major enterprises.Among them,adding storage nodes as an important operation of FastDFS to reduce system load,increase system throughput and disaster recovery backup is of great significance.However,in the source file synchronization task allocation which the new storage node must complete,the method adopted is too simple and inefficient,and it is easy to cause load skew and performance bottleneck under the existing service load.Therefore,based on FastDFS new storage node synchronization task allocation problem,this paper proposes LBB-STA algorithm to complete the multi-node load balancing calculation.The calculation results serve as a reference standard to consider the old storage node synchronization task allocation scheduling priority to perform the new storage node synchronization task,and cooperate to complete the new storage node synchronization task combined with the multi-old storage node concurrent task execution mechanism.This algorithm takes the improvement of the original FastDFS new storage node synchronous task allocation algorithm as the innovation point,draws lessons from the classical load balancing algorithm's thought and application environment,fully considers the performance and load of the old storage node,calculates the processing performance margin by periodically collecting the state information of the storage node,and processes the data in groups according to the processing performance margin rate,and then equalizes them in high groups.The sub-synchronization task is divided into several old storage nodes which are assigned synchronization tasks.The asynchronous threads are opened in parallel and the source files are synchronized to the new storage nodes.LBB-STA algorithm takes full account of the real load and performance tolerance of storage nodes in practical work,and allocates synchronization tasks to storage nodes reasonably and equally,so as to improve the utilization of system resources.Finally,the LBB-STA algorithm is programmed and implemented,and two groups of comparative experiments are designed to compare the synchronization time and the load rate of storage nodes between the original algorithm and the LBB-STA algorithm.Experiments show that the LBB-STA algorithm gives full play to the performance of storage nodes,greatly reduces the file synchronization time,and has better load effect.