Design And Implementation Of A Distributed Video Stream Storage Metadata Server

In recent years,because of that public security threats become increasingly serious,it's urgent to apply the security monitoring technology for public security.Traditional security monitoring technology used to store camera datas by ordinary distributed storage systems.However,most of those systems used multiple replicas to ensure the reliability of datas and result in low utilization of storage space.Moreover,they used the random scheduling algorithm to dispatch storage node,and ignored the load balance of storage node which led to jittering of nodes.In addition,those didn't consider the effective time of video streams during recovering so that the value of recovered datas was low.The system of distributed video stream storage implemented a metadata server which combined distributed storage architecture technology with security monitoring industry.It aimed to improve the reliability of monitoring datas and system scalability.On the other hand,it supported that users can access any camera at any time within the legal authority through the Internet.The metadata server,which combined with the timestamp strategy,designed the distributed storage system based on erasure coding and the volume information management structure,furthermore,accomplished the balance of scheduling of the storage and recovery tasks.The specific research contents are as follows:1.Timestamp policy was designed and implemented to maintain the consistency of volume information among different servers.Each server recorded the timestamp of all the volume datas in the system.And the metadata server was the only reference standard to judge effective time of the volume information for others.2.Metadata structures of two dimensions that were indexed by nodes and volumes were designed and implemented.The metadata server maintained the system state of each storage node for the task balancing scheduling.At the same time,the metadata server recorded the distribution of the data on each storage node and the information about the erasure coding strategy for the volume datas that used to provide basis for the recover task once data was lost.On the other hand,the system supported the dynamic expansion.3.Task scheduling strategy was designed and implemented according to multiobjective algorithm.For the distribution of storage tasks,the objective functions not only guaranteed the load balance of the nodes,but also minimized the difference about the total number of tasks with the nodes.For the priority of the recovery task,the objective functions were preferred the volume which had lower redundancy,higher loss rate,more longer remaining active time to raise the reliability.At last,through the testing of function and performance for the distributed video stream storage system,the result indicated that the metadata server can meet the control and management requirements of storing,reading and recovering video streams.In addition,the scheduling strategy of storing and recovering not only reduced the load imbalance probability of the storage nodes,but also delayed executing inefficient recovery tasks to make the use value of restoring data increase.