Study And Implementation Of Key Technologies In Cloud-based File Synchronization System

As the use of Cloud Storage services, such as Kingsoft KuaiPan, Dropbox andEverNote, springs up rapidly, Cloud Computing emerges in people’s daily life. The keytechnology of highly efficient Cloud Storage application rests in synchronization ofuser’s data among multi-devices. The Storage Service system will withstand morechallenges with the gradually growing number of users, accompanied by the surgingamount of requests in parallel simultaneously, and explosively increasing demand ofstorage capacity. Only safety-reliable and scalable Cloud Storage system can guaranteethe quality of service. Openness and co-operation are the two aspects of the themes oftoday’s Internet and the openness of Cloud Storage system fulfills the trend.This paper has designed and implemented a safety-reliable and scalable CloudStorage subsystem called Cloud-based File Synchronization System (CFSS). A CloudDrive Client Software (CDCS) is designed and developed for CFSS, and an openRESTful API is provided as well. Moreover, the paper introduces the further research onthe key technique of file synchronization algorithm under cloud environment with theresult of multi-equipment file synchronization algorithm based on file’s metadatacollection. The major contents referred in the paper involve the following pointsgenerally.The overall design of CFSS. CFSS absorbs load-balancing and MongoDB clustertechnique, which improves the ability of request processing in parallel and the ability ofrunning normally even if some errors may occur. The design of CFSS has met thefundamental requirements for safety, reliability and scalability.Multi-device file synchronization algorithm. File synchronization amongparticipants in the cloud environment presents a star topology structure with the centralnode called Cloud providing cloud storage service, and a variety of devices connectedwith the cloud called Client. If there is only one Client, firstly, file changes of the Cloudor Client can be achieved by comparing the metadata’s collections of two correspondingtime points--the point when file synchronization operation was finished last time andthe point when file synchronization operation begins this time. Second, combine theCloud’s file changes and the Client’s files changes to get the final file changes. Third,apply final file changes to keep the Client and the Cloud in sync. The seemingly simplydescribed procedures are actually complicated and challenging in practical operation. Some neat abstract and anti-abstract ensure the whole process to obtain the file changeseasily. Further abstraction of exclusive occupation of the synchronization time, and theloosely coupled relationship among the clients make the synchronization operationbetween each Client and Cloud independent. As a result, the file synchronization ofmultiple Clients and Cloud can be regarded as a single Client and Cloud assumedAbove. From this aspect, the multi-device file synchronization problem is then solved.The engineering implementation of CFSS. The engineering implementationincludes two main tasks. One is to develop a RESTful web service program based onJersey framework, and the other is to develop a Windows version of CDCS.The deployment and testing of CFSS. Deployment means to apply the finishedRESTful web service program to the supercomputer Nebulae. All of the data havingbeen stored in ParaStor of Nebulae, testing means the performance test of theco-operation between MongoDB and ParaStor based on YCSB.Having experienced the practical testing in the engineering application, the CFSScan run normally in general.