The Research On Optimization Technology Of Data Access In Video Surveillance System

Video surveillance systems are playing an increasingly prominent role in the field ofsecurity as people pay more attention to public security. The modern video surveillancesystems need to store massive data including videos and small files, which including keyframes and moving object images. It is very important to access these data efficiently.Massive surveillance videos are stored and analyzed based on Hadoop distributedframework. There are two main problems existing during the process of accessing andanalyzing videos. One is that OpenCV doesn’t provide the interface of parsing the binarymemory stream, which can lead to a bad performance of video reading during thedistributed video process. The other is that HDFS has a bad performance in accessing alarge amount of small files while it is very suitable for the streaming access of big files.The interface of parsing binary video stream in memory is implemented byunderstanding the theory of video decoding of OpenCV, transferring the binary memorystream to the packet and decoding the data among packet, which can reduce theexecuting time of Map tasks and achieve the read performance promotion of video. Asmall file accessing module is added to HDFS using the characteristics of filecorrelations and access locality of small files among video surveillance systems. Thesmall files are merged into big files, two-level metadata mechanism is adopted to reducethe memory occupation of NameNode and make full use of DataNode’s memory, whichthe first-level metadata are stored among NameNode and the second-level metadata arestored among DataNodes. In addition, the cache management is adopted to improve theaccess performance of small files using distributed independent cache infrastructure, thehot small files are cached in DataNodes.Experiments show that the memory stream-based approach is10%~30%faster thanthe existing method. For the small files among video surveillance system, the writeperformance is57%~62%faster than the native HDFS, the read performance is10%~13%faster than the native HDFS and the memory occupancy rate of NameNodereduces by5%~12%compared with native HDFS.