A Design Of Client-based Video Deduplication

With the rapid development of the Internet and cloud storage,more and more users are storing local data in the cloud server to alleviate local storage pressure.Since users often store repeated data in multiple locations in order to enhance data reliability,availability,and recoverability,this will result in a large amount of repeated data being stored in the cloud server,especially for videos or images that take up a lot of storage space,which cause a lot of unnecessary server storage resources to be wasted and network bandwidth occupied.Therefore,in order to save server storage space and network bandwidth,client-based deduplication technology(referred to as “ deduplication technology”)has been widely used in the field of cloud storage services.At present,the existing video deduplication technology is basically based on traditional cryptographic hashing for accurate matching or using a short summary of key frames for deduplicated video detection,resulting in low deletion rate.On the one hand,for videos with semantic characteristics,they can still be regarded as repeated videos even after a certain content retention operation,thus deduplication technology for accurate matching will no longer be applicable to the deduplication of similar videos;The technique of deduplicated video detection using video key frames only considers the spatial characteristics of a video and ignores the temporal characteristics in it,resulting in low detection accuracy.On the other hand,the research on the proof of ownership in similar video deduplication technology is still in its infancy,and the existing proof of ownership for videos is also verified by the traditional cryptographic hash function.This proof of ownership can only be applied to video deduplication of the exact same content and no research has been found for proof of ownership for similar videos.For these above deficiencies,this thesis studies and analyzes the existing methods,in order to achieve more efficient and safer similar video deduplication,and has achieved the following results: 1?In order to solve the problem of low accuracy and deletion rate in video deduplication process,this thesis proposes a video deduplication approach based on spatio-temporal matching.On the one hand,our approach divides a video into shots using the shots segmentation algorithm to generates the spatio-temporal representative frame TIRI of each shot,and extracts the sensing features of the TIRI.By combining the spatial and temporal features of a video,the problem of low accuracy in deduplicated video detection is avoided.On the other hand,our approach improves the problem of low deletion rate in precise matching of repeated videos by using the image perceptual hash algorithm based on discrete cosine transform(DAN_phash algorithm).Finally,compared with the existing methods,our results show that the recall rate for similar videos can reach 87.50%,and the false positive rate for different videos is only 5.49% with the threshold set to 0.3281,better than existing methods.2?In order to ensure the security of the user data,this thesis proposes an approach for the proof of ownership of similar video based on random partitioning.This approach uses the multiple shots content of a video to generate the ownership challenge information,and the user needs to generate ownership evidence and upload it according to the challenge information sent by the server,to complete the proof of ownership process.In order to avoid the replay attack and the spoofing attack of an attacker who only has the perceptual digest sequence of a video,the approach introduces randomness to ensure that the challenge information in each verification process is unique.Through the security and proof analysis and the experiment results,the feasibility and effectiveness of this approach is proved.