Research Of Data De-duplication Based On Mobile Terminals

With the rapid development of the Information Society and the Mobile Internet, coupled with the rise of the service based on Personal Cloud, the business of the Cloud Storage and the Cloud Synchronization has been grown rapidly in recent years which make data explode on the net. As a result, it has put pressure on the space of network storage and the network bandwidth, especially in the field of the Mobile Internet. Because a mobile device has a smaller physical storage and the costs of access networks are relatively high, it is important to reduce the transmission and storage of redundant data.Data De-duplication Technology can reduce the data redundancy by the way of deleting the same file or the same data block in the Data Storage System. In this way, the space of the network storage could be saved and the utilization ratio of the network bandwidth could be improved. According to the different granularity of data operations, Data De-duplication Technology can be classified into whole file level, data chunk level and data object level. Nowadays, Data De-duplication Technology has been applied in many fields, such as data backup systems, data archiving systems and disaster recovery systems. Due to the low bandwidth and high-cost characteristics of the network through which we can access to the Mobile Internet and the relatively small physical memory of mobile terminals, Data De-duplication Technology have not been used wildly in the field of the Mobile Internet. There are also some examples of application for Data De-duplication Technology based on the Mobile Internet companied with some disadvantages. For example, the algorithm which is on base of the Fixed-size Partition (FSP) has been applied in systems of Cloud Storage and Cloud Synchronization in the filed of the Mobile Internet. Though this method is simple and efficient, it is so sensitive to the operation of adding or deleting data that the Data De-duplication Rate is lower than expected.Data De-duplication Technology is proposed to applied into the field of the Mobile Internet in this essay. Data De-duplication Rate and machine overhead should be taken into consideration to measure the algorithm when we optimize and apply the algorithm. A deal of work has been made with focus on the two aspects. First, the optimal Data De-duplication Algorithm based on the Mobile Internet was selected companied by taking Data De-duplication Rate and machine overhead into consideration. Second, some kinds of files were selected as the experiment samples to optimize and apply the Data De-duplication Algorithm on the basis of some restrictions of the Mobile Internet. What’s more, a Cloud Storage and Synchronization System was designed based on the Data De-duplication Technology.Data De-duplication Technology based on Whole File Detection Technique is impossible to detect the redundancy data within a file, making its Data De-duplication Rate much lower than data chunk level Data De-duplication. However, Data De-duplication Technology of data object level is so complicated to select the basic unit to operate data that it is not suitable for the application in mobile terminals. According to this, the essay focused on Data De-duplication Technology of data chunk level of which some key steps are optimized and estimated to make it accommodate to the filed of the Mobile Internet. So that the basis of the application for Data De-duplication Technology based on the Mobile Internet platform is provided in the field of Cloud Storage and Cloud Synchronization.