The Research And Design Of Massive Video Data Storage And Transcoding System Based On Hadoop

With the development and evolution of the Internet, the generated data is rapidly increasing and will be even expanded by stream media transmission, an important application of the Internet industry. Centralized storage of stream media data is too expensive while the integration of data storage and analysis is hard to realize. Various cloud calculation platform were published for massive data storage and processing. As a tool for distributed massive data storage and analysis, Hadoop is currently wildly utilized in non-video tasks of data storage and analysis like Internet and medical system. Due to the difference in file format, storage methods and search methods, the application of Hadoop in network stream media transmission system is highly challenged. This thesis realized the data storage and analysis of massive video data through the research on properties of stream media and Hadoop platform. This thesis maily completed tasks as follows:1. Analyzed HDFS and MapReduce platform and network stream media transmission system, accomplished the requirement analysis of massive video data storage and transcoding system and proposed design solution of the system.2. Completed the module design of stream media data storage, HDFS stream media data read and write, distributed transcoding, data base and Web management. Realized the Hadoop based integrated solution of storage, transcoding and management of massive video data.3. Analyzed the performance of data storage, download rate and distributed transcoding of the system. Research shows that storage strategy based on Hadoop is advantageous in quick search of massive video data, transcoding efficiency of distributed transcoding system and download and playback through general players.