A Study Of Image Deduplication Based On Perceptual Hash

With the popularity of cloud storage services,more and more data are stored in cloud servers.In fact,most of the data in cloud storage server are redundant,especially the multimedia files which take up more memory,such as image and video.In order to save storage space resources,network bandwidth and data management cost,client-based deduplication technology has been widely applied.Generally,client-based deduplication technology for image mainly includes two parts: duplicate image detection and proof of ownership(Po W).Duplicate image detection is the process of mapping images into brief abstracts according to mapping functions and then matching them.The "proof of ownership" concept was proposed to ensure the security of users' data.By the proof of ownership,the cloud server determines whether the client really owns the completely original file.However,because of the new application of digital media,the traditional client-side based data deduplication technology is facing new challenges for multimedia files such as images.This paper first studies the existing deduplication schemes and summarizes the following weaknesses of the existing solution:1.Due to the existence of semantic gap,the image similarity matching can't be completely accurate,which leads to image deduplication probability decreases significantly or image non-duplicate data is erroneously deleted.2.At present,the proof of ownership of duplicated image deletion for cross-clients is still in its infancy.Researchers have proposed a scheme for proof of ownership through random auxiliary images.However,there are still defects of wasting server space resources and network bandwidth in the scheme.For the purpose of making up for the above deficiencies and achieve secure and accurate cross-client image deduplication,this paper has done a lot of research on the accuracy of the image deduplication process and the security issues in the image deduplication,and the following are obtained results:1.In order to solve the problem of low repetition rate of image detection,a secure deletion scheme for repetitive images based on perception is proposed in this paper.On the one hand,this method extracts image features by perceptual hash algorithms to avoid the problem that the traditional duplicate data deleting technology can only delete the exact same images,which leads to a very low probability of duplicate image deletion.On the other hand,the method extracts image features from different aspects by two perceptual hash algorithms and implements dual detection through image dual features,which reduces the probability that the image non-duplicate data is erroneously deleted due to the lack of sufficient image information through a single image feature.Finally,the experimental simulation results show that,by setting appropriate thresholds,the proposed scheme can still guarantee high deleting rate of image duplicate data in the case of reducing the error deletion probability of non-duplicate data compared with the existing scheme.2.In order to solve the security problem in the process of duplicate image deletion,this paper proposes a scrambling-based image ownership verification scheme.The scheme uses the query information sent by the cloud server and the original image itself to generate the ownership evidence without the aid of auxiliary images.Thus,the scheme can save the storage space resources of the cloud server and network bandwidth.According to the results of security proof and simulation experiments,the scheme not only saves storage space resources of the cloud server and network bandwidth,but also has obvious advantages in terms of calculation time overhead.