Design And Implementation Of Distributed Video Storage Fault-tolerant System

Today,the video surveillance system is increasingly used in daily life,it brings convenience,but also leads to massive video streaming data storage demand at the same time.Distributed storage system can meet the demand well,and because of the possibility of failure,fault tolerance must be reflected in the design of this system.Compared with the typical distributed storage system architecture at home and abroad,it is not difficult to find that system availability,data reliability and consistency,system efficiency,are usually the problems such the system needs to solve.Therefore,this thesis,based on this background,through the relevant knowledge points and theoretical learning,designs and develops a distributed video stream storage fault-tolerant system,the main features of it are:1.Applying the RS erasure codes.On the basis of the flattening of the whole storage space,the system proposes the semantics of the data block and the volume,converts the streaming storage into block storage,and determines the placement of data blocks through the specific erasure-coding strategy,so that the fault tolerance of data by the way of erasure-coding can be achieved.2.Proposing a hybrid redundancy fault-tolerant strategy;the system regards the frequently accessed metadata as "hot" data,used a copy of the form of storage,the video stream data as "cold" data,used the erasure-coding of the form of storage.In order to accelerate the access of metadata,and improve the utilization of storage space at the same time.3.Using timestamp comparison mechanism;it not only ensures the data operation consistency of the system among the different modules,but also avoids the effect of single point,due to the reduced access of data amount and times.4.Implementing the final consistency of operating scheme;the system regards the multi-step instructions as transactions,executes the command queue by the way of non-blocking to balance the system availability and consistency.5.Proposing the file block version number recovery mechanism and the optimal data recovery model.By reducing the unnecessary data recovery and determining the order of optimal data recovery,improving the efficiency of data recovery to a certain extent.In addition,in order to achieve scalability better,the system uses a hierarchical design architecture,transiting network to the business process,and then to the file system.Also,by using the programming framework of Epoll and thread pool,that is to say,the corresponding thread and thread pool execute blocking erasure-coding operation and disk I/O respectively,thus,the speed between CPU and disk can be balanced.Through the test,it can prove that the distributed video stream storage fault-tolerant system designed and developed in this thesis not only can correctly store and manage massive video stream data,ensure data reliability,enhance system availability and expansibility,but also improve the efficiency of the system to a certain extent through the corresponding optimization strategy.Therefore,this thesis is of great significance to the development and research of distributed video stream storage fault tolerance system.