Research On Load Balancing Of Distributed Storage System For Video Surveillance

In the era of big data,distributed storage systems are an effective way to cope with the surge in data.The load balancing of the system greatly affects the performance of distributed storage systems.Existing load balancing strategies basically generate index metadata for storing data storage locations,which greatly affects the performance and scalability of distributed storage systems.In the video surveillance scenario,write request data needs to be sliced and saved,and the large amount of index metadata generated will lead to a lot of waste of storage space.Therefore,in the video surveillance scenario distributed storage system,it is suitable to use calculation to allocate requests to achieve load balancing.Consistent hashing and its improved algorithm avoid the storage of index metadata,but the method of dynamically adding and deleting virtual nodes on the hash ring to achieve load balancing will generate a lot of data migration and identification overhead.The thesis uses the characteristics of more writes and less reads in the video surveillance distributed storage system,and proposes a virtual node state management strategy.By changing the virtual node state value to avoid changes in the hash structure,it achieves a balanced distribution of read and write requests while reducing unnecessary data migration and identification overhead,and improves the overall performance of the system;Proposed a management strategy for the number of virtual nodes,considering the available storage capacity of the server,and designed a calculation method that more closely fits the law of server load changes,to further improved the balance of load distribution;A server grouping strategy is proposed to facilitate the management of data copies,to ensure that when a server is down,the location of other copies can be quickly found,reducing the overhead of data identification.At the same time,the server with the lightest load can be selected from the group to process read requests,further improving load balancing effect.For the video surveillance storage system,a dynamic consistent hashing based on group(DCHG)is implemented.The test results show that DCHG achieves a better load balancing effect and avoids a large amount of data migration and identification overhead.Compared with the existing work DWCH and DCH,the effect of DCHG balancing load is26.21%?37.77% higher than DWCH,23.27%?34.76% higher than DCH,and the effect of balancing capacity utilization is 63.54%?72.12% higher than DWCH,63.13%?71.96%higher than DCH.