Research On Load Balancing Problem In Distributed File System In Cloud Computing Environment

Cloud computing is the future direction of development of computer and communication technology. And the cloud storage technology, represented by distributed file system, is an important part in cloud computing. Load balancing technology is a core technology of cloud storage. In recent years, Hadoop Distributed File System (HDFS) with its high availability and scalability characteristics obtains of great concern in the industry and is the commercialization of research focus. Therefore, the practical application research on the HDFS cluster has very important significance.Because of its significant advantage, HDFS has become the current mainstream of application architecture. But due to a single evaluation and a single load balancing threshold strategy, in the practical application of large web sites, if faced with highly sudden and random concurrent access, HDFS can not deal with. And there will be paralysis of the status of the server system. Therefore, the paper try to address how to design load balancing strategy for the HDFS adapting to this particular scenario.On the basis of the analysis of the HDFS core code, paper proposed the improved algorithm, which used threshold strategy based on load migration. Firstly, paper described the evaluations which has been chosen depending on the application scene and showed the way how to caculate the evaluation index. Secondly, the paper analyzed the form of the evaluation function, and based on the specific application, indicated the effectiveness on the selection of the weighting form evaluation function. And then, the paper introduced the relevant methods in the multi-attribute decision theory. Through combining with calculation of weight vector in the load evaluation function, the paper ultimately determined the evaluation function. Finally, against the significant deficiencies of the single threshold load balancing policy for HDFS, the paper proposed a double-threshold policy and defined the server load migration program, which completed the design of the improved algorithm.The paper not only presented the improved algorithm, but also made it in HDFS by programming. And then, by designing the performance test, the paper verified the effectiveness of the improved algorithm.The analysis results show that the improved load balancing method can help HDFS to assigned highly strength and random load to server group in a balancing way, so that the system can avoid a single server overloading.