Research And Application Of Hadoop-Based Offline Video Data Processing Technology

Nowadays, smart city becomes a basic aim of city construction in the age of information. One of the most important is intelligent security monitoring. Video surveillance systems have been widely used in all walks of life, so surveillance video data has become a class of big data The traditional video monitoring mode, including collection and playback, has gradually cannot meet the demand for video surveillance, besides we want to extract the useful information from videos for providing security service effectively. Efficient processing of a large number of surveillance video data has become an important research topic.Firstly, the paper deeply analyze three important parts of Hadoop frame, including HDFS、MapReduce and HBase. And I summarize some current popular content-based video processing method. I descript some bottlenecks and deficiencies of the method of video processing. According to the analysis of video processing characteristics, I proposed and implemented a distributed offline video processing method based on Hadoop MapReduce framework. By designing video processing methods and interfaces, the improved Hadoop MapReduce can process video data as text file and binary files. Thus developers no longer consider the issue of Hadoop MapReduce cannot directly process video data when process video in parallel based on Hadoop MapReduce, thereby developers focus more on core video processing algorithms.I discovered in experiment that video processing time associate with video complexity. For this feature, I redistribute the distribution of data on Hadoop HDFS for optimizing overall system performance when Hadoop MapReduce processing video.The paper further analysis and design Hadoop MapReduce data-types. Based on the this, I implemented three applications including distributed video transcoding, distributed video abstract and distributed searching.The experimental results show that using Hadoop MapReduce to process the mass video data can reduce the time of video processing. Redistribution of HDFS can reduce system I/O and further improve the efficiency of Hadoop MapReduce video processing.