Research On Technologies Of Load Balancing In Distributed Storage Based On Hdfs

The changes in online education result in a rapid growth of users and video resources, which challenges the storages as well as the overall performance of the system. It’s a key issue that how to properly store data and efficiently return the required data to the user. Distributed technology, one of the most current popular technologies, is the solution to the problems above. Especially, load balancing is the key algorithms affecting system performance. After the comparative analysis for a variety of algorithm about load balancing, this research improves this algorithms. In addition, this research designs and implements an efficient distributed video storage system.Firstly, this research designs a load balancing evaluation equation based on multiple indicators to deal with video files which needs to be fully fit for all aspects of computer performance characteristics. Then, this research proposes a load balancing algorithm based on the real-time work of a node and the equation.Experiments show that the implementation of new load-balancing mechanism can greatly enhance the overall resource utilization of a distributed storage system,so that all nodes in the system to achieve a more perfect balance and to ensure the quality of service request by the client. In addition, the research also improved a resource scheduling by delay policy in the special scenes for processing videos, for a serious decline in system performance if often brings the large-scale transfer videos, so that the algorithm can dynamically adjust the waiting time. Experiments showed that, both overall system performance and job response time had been greatly improved.Finally, this research designs and implements a distributed storage system based on Hadoop. What’s more, this system makes an improvement for the characteristics of video files. The advantages are as follow: 1) when uploading video files, this system chooses the optimal node to store the data according to the real-time load conditions of all nodes. In the meanwhile, the system avoids the influence that caused by the operation of load balancing on the quality of service requested by users. 2) When the users require video files, the system tries to ensure local data services, so that can improve the overall performance of the system. In addition to ensure the quality of service, the system takes into account the response speed of the users’ request.