Research On Distributed Storage And Retrieval System Based On Hadoop For Massive Video

With the widely use of recent monitoring equipments in various types of security field and the rapid growth of the surveillance video information, how to store and retrieve the massive surveillance video data effectively has become a hot issue in the research field. For the demanding of surveillance video storage, after having done the research on the surveillance video retrieval technology and distributed processing platform, this thesis designed and implemented a massive surveillance video storage and retrieval system based on Hadoop. This thesis has mainly studied the following two problems:On the one hand, based on the content retrieval process of massive video, this thesis analyzed all kinds of data which needs to be stored in the retrieval process and related storage requirement. Besides that, according to the video storage module, abstract video storage module and target image storage module this paper carried out the detailed design and implementation. In each module, the high throughput data such as video uses HDFS distributed storage system for storage, for the data which needs classification of storage and real-time data is read column-oriented database HBase. For data storage and processing module exchange between an XML file to resolve the differences between platforms and applications, and finally through the experiment proved that read and write data memory module in the system to read and write performance efficiency.On the other hand, etrieval performance is also very important besides the massive data retrieval accuracy of retrieval for user. Since the image feature extraction and matching process is a time consuming process and performance, taking into stand-alone treatment programs in this regard is insufficient. This thesis introduces the idea of distributed processing, using MapRedeuce distributed programming model in Hadoop systems to finish the target image feature extraction and feature matching video retrieval operations. At last, experiments verified by processing on a distributed processing platform better than traditional stand-alone.The experiment prooved that it performed much better on the distributed processing platform than on the traditional stand-alone platform.