Data Reduction Method Via Provenance In Massive Video Sharing System

With the emergence and growing popularity of video-editing software and video sharing systems, the number of videos on the Internet is increasing rapidly. Near-duplicated videos created by video-editing software generally exist in the massive videos, which increases the network traffic bandwidth overheads between users and video sharing systems. The access frequency of videos follows the Parato distribution,and a few of the hot videos account for most of the visits while a large number of cold videos only account for few or no visits. Moreover, the video sharing systems can't tolerate any user's data loss, which results in the significant cost of storing massive cold data for long period. Hence, it is a challenge to efficiently reduce upload network bandwidth and storage space of cold nearduplicated videos but not affect the user experience and cause the data loss.A video data reduction scheme via provenance called Dpbvd is designed and implemented to reduce the storage space in video sharing system. It can avoid the video upload and save network bandwidth by transferring the video-editing operations to the cloud server when editing videos existing in systems. In the meantime, the scheme can reduce the total management overhead of system by deleting the cold near-duplicate videos having provenance information in the server. In the client, the scheme collects the video-editing operations and videos information which can be used as the application-layer provenance of new-created videos when users edit videos, and then sends them to the server. In the server, the scheme records the cumulative account information and the last recently stage account of videos, and dynamically predicts the hot degree of videos based on the video access popularity trend and the recent visits to videos. The server caches the hottest videos to enhance the video accessing speed. For the cold videos, the scheme scans the provenance of videos to find the cold videos with provenance information, and then deletes them to save the storage space. The deleted videos can be still accessed after being regenerated by their provenance. Hence no video data are lost in the scheme.Experimental results demonstrate that the scheme can efficiently reduce the space cost, and precisely forecast the video hot degree, while has little effect on the users' access speed.