Research And Application Of Video And Image Processing Based On The Hadoop Technology

With the rapid rise of various social networking sites, as well as the rapid development of mobile devices, video image and other unstructured data have appeared an explosive growth. Video and images have become an important way for people to exchange information. Moreover, the underlying information within digital images, online video and other unstructured data has become a major source of information for various research fields, such as national defense, military, industry, agriculture, information, medical and so on. However, how to efficiently dig out the potential value from the mass of images and video data has become a research focus in Data Mining so far. Furthermore, H.264 and JPEG2000 have become an extensive compression standard of video and still images to date. Therefore, preprocessing of digital image and coding and decoding of video data in the distributed cloud platform are the precondition to explore the potential knowledge from these vast of unstructured video image data.This paper focuses on the topic of "Video Image Processing Based on the Hadoop". In this work, the origin, the meaning and the development status of Hadoop technology were elaborated. In addition, the difference between cloud computing and other high-performance computing, i.e. parallel computing, grid computing and volunteer computing was also analyzed in this article. Finally, a Hadoop platform had been set up on the Linux system.Operation principle of the Hadoop cluster and some experiments of image processing were mainly studied in this paper. These experiments of image processing, such as the grey value, the binarization, the edge detection, image matching for a digital image, were carried out on both the Hadoop distributed platform and traditional single platform. What’s more, a decoding experiment of network surveillance video with the SDV format was carried out. A type of video data was designed by using Xuggler visual database in this experiment, which can be used on the Hadoop platform within a cloud computing environment. The incomplete frame and lack of the key frames and less information about the head data can be solved in this experiment, when video was directly segmented on the Hadoop platform according to the given data block size of the platform. Also, the advantages and disadvantages of the traditional stand-alone decoding and distributed decoding were compared in this work. The results of this paper can be provided the theoretical and experimental basis for further research on the video image data on Hadoop platform.The innovation of this paper are:1. The algorithm transplant of the video image processing based on the Hadoop technology has been studied. The platform is completely free and open source, and the nodes are free to scale-out without any additional cost of the original platform. Therefore, that platform are more suitable for the distributed storage and the intelligent computing of the growing video, image and other unstructured data in the Big Data Time.2. Surveillance video with the SDV format has been decoded on the Hadoop platform. A small video consolidation method has been designed, which is suitable for HDFS storage mechanisms. Compared with the traditional stand-alone test, this method can take advantage of the Hadoop distributed computing, that is, the larger the amount of data, the more obvious the increase of decoding efficiency.