An Educational Resources Management System Based On Data Deduplication

With the advancement of informatization for basic educationin China,digital construction of educational resource is continuously strengthen, and educational resource centers at all levels arealso be successively built. And due to uninterrupted operation of educational resourcemanagement system, amount of data in the system rapidly rises. A trivial approach to resolve this problem is that increase the number of storage equipmentand improve the network bandwidth, but this can led to developmental crisis of the system which is caused by high cost in system operation. However, through our research, there are a large number of duplicate data within educational resource, also it occurs between all kinds of educational resources, and it is unnecessary to store and manage these redundant data. Thus, the optimal solution is adopt wipe technology for redundant data to Educational resource storage and management. And this approach can radically reduce the growth rate of data and the cost in system operation, thus promote the construction of educational resource center.Aim to radically reduce storage space occupancy, we design and implement an educational resource management system based on wipe technology for redundant data. Our system adopts popular B/S framework, which includes four functional modules, namely education resource management module, data deduplication module, user management module and integral management module, and each of module includes multiple child function points. At data deduplication module, we propose an improved data partitioning algorithm based on CDC. And we adopt MD5 to this method to calculate fingerprints of data block. Moreover, bloom filter technology and memory file mapping technology are applied to our system for increasing the detection of repeated data block. At education resource management module, our system can provide a variety of data retrieving method to improve query efficiency of resource, such as precise retrieve, fuzzy retrieve and full-text retrieve. Additionally, we Implement effective incentive measure for our system, and this measure can arouse enthusiasm of data uploading for resources builders, thus can ensure the stable and high quality of data source.As for experiment, we test the function and performance of our system. And the results show that our system can normally operate under practical environment, also can ensure consistency of data and integrity of resources. With the redundant data of our project, we test the rate of data deduplication in our system. Experiment shows that our system can radically save storage space and reduce bandwidth of network, thus can save costs of operating system and promote the development of educational resource center.