The Design And Implementation Of Cloud Storage System Based On Hdfs

In recent years, cloud computing is becoming popular because it can handle large amounts of data quickly. The storage systems of large capacity and high performance are the foundation for cloud computing. Among them, the distributed storage system named HDFS is the most well-known open source implementation of Google File System and it has been widely used and studied. It provides a good reference for the major companies and research institutions to build cloud storage services. However, HDFS, as an auxiliary system of cloud computing, still remains some deficiencies such as inefficient issue with small files. These deficiencies led HDFS not to providing a direct application for business needs. To solve these problems, this paper proposes a novel approach for small file storage and implements a distributed storage system based on HDFS cluster.The system designed by this article includes two access modes:Web mode and NFS mode. This paper focuses on the small file storage, user management, quota management, NFS and HDFS integration, etc. The main work is as follows:1. This paper proposes a novel approach for small file storage and we set a series of simulations.2. Design a complete set of solutions for the user demand from enterprise. This article gives a very detailed description of the design of client, the architecture of the system. Also, this article describes the demands and the functions in detail.3. This paper completes three implementations. Firstly, we design and implement the storage system based on Web mode. The main achievements are user login, quota management, file sharing and so on. Secondly, we introduce NFS into cloud storage systems. The users can access their own space and manage their quota by NFS. Finally, we completed the management system.4. For the two modes of the system, we use black box technology to design very detailed test cases to test their performance. We compare the performance of uploading files, downloading files and uploading files under different scenarios.In this paper, the simulation results show that the new approach has improved the efficiency of storing massive small files and we design and implement a distributed storage system which meets the business needs. We successfully introduce NFS into HDFS. In the actual test and application, the system owns comprehensive and practical functions and also performs well.