Study On Storage Mode For Mass Data Storage

With the development of Internet technology and the digital era, the amount of data showing a move terabytes of growth trend, the storage system is facing severe challenges, and data storage has become a bottleneck restricting the development of enterprises. To solve this problem, a growing number of enterprise data storage separated from the separate research and development, hoping to find a both efficient storage companies growing data, but also has high reliability, scalability and cost data storage solutions.As a storage system of cloud computing, cloud storage gather a large number of storage devices in the network to work together to provide storage services. Cloud storage from the emergence of the support of many businesses, continue to be applied and research, the maturing of cloud storage solutions, if applied in the mass data storage needs of enterprises will have a high feasibility.The content is the research of mass data storage architecture, which base the HDFS. The target is seeking to build a storage system that can solve the problem of corporate mass data storage by the study of the mature data storage architecture, and solving the storage problem by constructing a data storage system. The Hadoop Distributed File System is open source implementation of Google’s storage technology GFS, with high reliability, high-disaster recovery and expansion characteristics, based on the distributed low-cost storage devices, reduce the cost of doing business, and can storage mass data.First, this paper describes the knowledge of traditional data storage architecture and analysis to identify the scope of application of traditional architecture that cannot solve the problem of the modern enterprise storage; then describes background of cloud storage, system architecture, and Google’s cloud storage technology to understand the functional structure of GFS and Bigtable, contains part of the storage mechanisms and data management strategies; Then, in the base of the study of Google cloud storage, the paper study the HDFS which is the open source of it. We stressly analyze part of the functional properties of its storage principle, data communication mechanisms and implementation techniques in detail, All of them provide technical support for the further thesis research; Finally, based on the above research, We build a massive data storage system based on HDFS storage systems, and analyze system structure, principle, process and system feasibility analysis.In this paper, we build a mass data storage system based on HDFS, for solving mass business data storage, building a private storage system based on cloud storage, provides an effective solution. In the same time, in order to achieve the full integration of the businesses and HDFS architecture and solve the problem of low efficiency of the HDFS storage of small files, the architecture has been improved which contains a mechanism of the small file merging as a separate layer. With the HDFS, the enterprise can flexibly build a private storage system.