Research And Application Of Distributed Storage Technology In Video Surveillance System

In the era of big data,with the rapid development of large-scale data centers and cloud services,as well as green development and energy conservation and emission reduction led by the "dual carbon" goal,the cost of massive data storage has become a key problem that enterprises need to solve.Video monitoring systems generate massive data,which needs to be stored for more than half a year,or even longer.However,the frequency of reading is very low,and the real-time requirements for reading can be hourly.At this point,a low-cost storage medium is needed to access these monitoring data.The high-density laminated disk can meet the storage requirements of massive video surveillance data.The monitoring system first stores the video data on the open source distributed storage system Ceph.In order to meet the requirements of front-end monitoring equipment for fast response storage,the Ceph distributed storage system uses highperformance disk Nvme SSD for storage.Then,the Snapshot Server will package the data according to time,place and device,and store the data in Blob format to the Archive Storage system,where Archive Storage is responsible for the storage,indexing and CMR disk management of archived data.The paper introduces the structure of the laminated disk in detail.The host management mode is used to manage the zones in the SMR disk,and 1% of the CMR zones in the SMR disk are used to store all metadata information.99% of the SMR zones are laminated,which are used to store data.The Archive Storage system is built with laminated disks.The disk density of a single cabinet is increased by powering up and powering down the disks.Theoretically,a single cabinet can manage up to several JBODs.Each JBOD has 107 disks.It provides services through disk groups.Each disk group can perform independent power on and power off operations,breaking through the limit of cabinet power.A cabinet can mount disks with higher density.In the study,the number of online disks is set to 1/3 of the total disks in the cabinet,so the disk density is increased to 3 times.The paper discusses the power on/off algorithm of the disk in detail,and designs the algorithm according to the power on characteristics of the disk to support the emergency power on/off reading of the disk.Finally,in order to improve the disk throughput,the powered on disk groups are divided into "read disk groups" and "write disk groups" for decoupling and independent processing.In this way,you can control that there are only read requests or only write requests in a single disk.Because the Archive Storage system in this study stores data in the smallest unit of Zone,data writing and reading are sequential.This mode reduces disk read/write seek switching and data sorting and relocation,which greatly improves disk throughput.The experimental results verify that the maximum write throughput of the single core CPU is increased by 32.71% and the maximum read throughput is increased by 18.48%.Compared with the existing imbricated storage system,this paper can effectively play the bandwidth advantage of sequential read and write of imbricated disks,but at the expense of the flexibility of data access,the cost of data modification in the archive storage system is higher.At the same time,this is also applicable to some special data storage scenarios,such as audit data,important data backup,monitoring video data storage,etc.