Key Technologies Research In Cloud Storage

In the modern era of high-speed growth of data, traditional database has been unable to meet the needs of dealing with large-scale data.Although some large cloud service providers such as Amazon, Google has been got a great success in this area, due to difficult to overcome privacy and data protection issues, many users still cannot transfer their applications to the cloud. The private cloud provides the most effective control of data security. Therefore, there is theory and application value in researching private cloud storage architecture.As the open source implementation of the Google File System(GFS), Hadoop file system HDFS actually become the standard of the study on cloud computing and cloud storage and the implementation of cloud applications and cloud services. However, HDFS architecture has a number of shortcomings, typically including lack of support for small files, performance bottlenecks because of the single NameNode.In order to solve the problem above, this paper focuses on the following contents:Based on the study of existing HDFS, comprehensive consideration of the universal cloud storage system characterized, take the improving storage efficiency as the goal, propose a cloud storage system implementations.The background of this research is to build a cloud storage system which used in the college environment, the characteristics of this platform contains a larger number of small files, while the existing HDFS process for small files is very inefficient. This paper draws on the treatment of the small files in HDFS with relevant literature, at same time, consider the idea of the two-dimensional packing algorithm, designed a small file processing algorithms in the HDFS file preprocessing to improve the response speed of the system.Build experimental platform with Hadoop0.2.3and design experiments to test and verify the performance of Hadoop-based storage and the ability of handling small files. The proposed method improves the utilization of resources and system response speed effectively.Raise a cloud storage system prototype. This system separates the contents of the object and the object metadata, design a rational and efficient metadata storage space and metadata man-agement services. For good compatibility, Amazon S3container-object oriented interface style is used. Give a multi-region architecture model; meet the system scalability, operability, data per-sistence and reliability requirements.