Design And Implementation Of Massive Data Storage System

Internet technologies are increasingly developed, and the amount of data on thenetwork are increasing explosively, so data management and maintenance, for businessesor individuals, are faced with enormous difficulties, not only cost is high, and there arehuge problems in the aspect of the reliability and availabilityof the data. The GFS systemof Google, was firstly given a complete and effective massive data storage theoreticalframework, and for large distributed data intensive applications, based on a large numberof heterogeneous storage machine based cluster, realize the high performance service ofthe mass data storage and reading, and provides the system excellent fault tolerance.Based on the theory of Google File System, combined with the characteristics ofhigh reliability of mass data storage, and heterogeneity in data storage nodes, this paperpropose the load calculation model based on machine storage performance and datablock reading times factors, and give the node choosing optimizing strategy of datareading, data writing and load balancing, at the same time, compare the optimizationstrategy with default strategy on data storage performance.Based on the detailed analysis of mass data storage theory, method and technology,this paper design the overall framework of the system, and subdivise function of eachpart of system, then get the function design of storage system, and realizes all function ofstorage system, including massive data storage, cluster node management and the opti-mization of node choosing strategy, finally, execute system test.The realizing system can effectively complete the basic function of mass data stor-age, and provide a certain degree of fault tolerance. The data reading and writing perfor-mance of system is good, and can provide high bandwidth services for user. The opti-mizing node choosing strategy based on basic storage service, is more balanced than thedefault policy on system load balance, and can increase the the speed of reading data in acertain extent.