Research And Implement Of The Massive Small File Storage Of The Web

This thesis studies a web-based solution for mass storage of small files. Digital CampusResource Management System is used as an implementation for the mass storage of smallfiles.This paper analyzed the current status of mass storage of small files and studies the basisof the mass of storage of small files. By studying how disk works, RAID technology,distributed file system storage as well as other related basic knowledge, this paper have aclear understanding of the overall storage architecture.In the light of web-based mass storage of small files, this paper utilized the stored portionof the Digital Campus Resource Management System as concrete storage implementationobjects. According to the GFS and HDFS and other core understanding of advanceddistributed storage architecture, as well as the accumulation of storage hardware basis, thispaper raises a web-based mass small file storage system. The system utilized weightedconsistent hashing as the algorithm of scheduling memory node and the overall loadbalancing. This paper also uses small-scale cache of the frequently used algorithm anddatabase to maintain the metadata information of the entire system. At the same time, aconfiguration file is used to manage all of the storage nodes in the storage system. Metadatamanagement and overall scheduling module concern information through monitoringheartbeat information of the module. The entire system has architecture of loose coupling andis able to implement different storage requirements on different levels.