The Application Of Deduplication Technology In Picture File System

Introduction of the author:Li Yinggang, male, was born in Dec,1986whose tutor was Professor Cheng Zhiwen. He graduated from Chengdu University of Technology in Computational Mathematic major and was granted the Master Degree in June,2013.Papers based on the author’s development work in the project "realize the function of physical tape library on disk as virtual tape library", combine autor’s think about storage area, do more systematic exploration of mobile terminal digital information structure, the deduplication technology development direction and the functional design development of storage systems with deduplication. Archive an easy-to-deploy, cheap, based Linux server image storage system with deduplication.Deduplication is an essential technology in the field of data storage backup. With continuous progress of the digitization of information, people need to store and transfer the amount of data will be more and more, data information structural diversification will be more obvious.3G or4G popularity of wireless networks to further promote the restructuring of the mobile terminal information from the mobile terminal, The trend of people need to get richer data information has become increasingly evident, this rich data information needs stimulate the body of the message text into pictures, and even further into the video.The paper first introduces the key technologies of deduplication and the problems have to face while its practical application, and then get the special needs of ultra-large number of pictures store, which different from general data storage. Depthly discuss the promble faced by the application of deduplication in picture storage, focus on content of logical structure about an open, easy to implement, and widely compatible image storage system, the server with Linux operating system, as the development and deployment platform of image storage system, which more than 90%share of the server market. Use the broad support and compatibility of the VFS, development a VFS-based image storage system. For the characteristics of the image files, choice the CDC algorithm for chunking image files, and think about the ext3file system to set the parameters of the CDC algorithm. Access problem for over a large number of image files, using file names hash value to speed up file search.Paper source insight for the project code management tools, gcc code compilation tools, combined with the picture file storage access operations to the file instead of the data block of the characteristics, the direct use Linux VFS file system file operation concept enablesa picture easily deployed deduplication storage systems. The storage system is based on the dynamic link library exists in the form of image storage and access required for basic operation interface, and can be easily extended to other functional interface according to the needs of practical application.