Research And Application On Data Distribution Strategy In Surveillance Video Cloud Computing Platform Based On Hybird Storage

The popularization of video surveillance system has contributed to the explosive growth of surveillance video data,distributed processing of surveillance video data has become a trend.There are many I/O operations in the processing of intelligent video analysis,and it is necessary to build a surveillance video offline distributed processing system with hybrid storage architecture using SSD and HDD.The traditional surveillance video offline distributed processing system mostly takes the number of the storage resources as the constraint to optimize the data distribution of cluster,without considering the storage media heterogeneity of cluster,as well as the dynamic of available storage resources and node load in the distributed processing of video tasks.Using traditional data distribution approach for large-scale video processing will lead to the low utilization of the high-performance devices and unbalanced cluster load,further defer the overall video task completion time.This thesis focuses on how to optimize the distribution of surveillance video data for surveillance video offline distributed processing system based on hybrid storage architecture and realize a high-performance surveillance video offline distributed processing system.Firstly,we propose a Processing Time Prediction Model(PTPM)for surveillance video data blocks based on studying current mainstream intelligent video processing algorithms.The PTPM integrates the characteristics of video data,the storage and computing power of nodes,and it can predict the processing time required by the video processing tasks at different nodes.Secondly,based on the PTPM model,we propose an Initial Data Distribution Strategy(IDDS).The IDDS takes use of the heterogeneity of node storage media and the difference of node load,and can minimize the initial load difference between nodes while stratifying the storage resources constraints.Thirdly,based on the dynamic characteristics of available storage resources and loads in the distributed processing of video tasks,we propose a Load Aware Data Migration(LADM)strategy.The LADM strategy can re-distribute the video data of cluster to reduce the load difference between nodes and further improve the node storage resource utilization.Finally,we implement a Docker based surveillance video offline distributed processing system,and conduct the extensive performance experiments for verifying the proposed IDDS and LADM strategy.The experimental results show that our methods can effectively improve the resource utilization of SSDs,ensure the load balancing of clusters and further improve the processing efficiency of surveillance video tasks.