Design And Implementation Of Campus Cloud Storage System Based On HDFS

With the constant development of campus informatization,Colleges and universities information data storage problems emerge rapidly.However,the existing data storage platforms can't meet the growing demand of massive data.Therefore,it appears to be extremely important to study and develop an effective storage platform to store these massive information data.Cloud computing is the increase of the related services based on Internet,use and delivery mode,usually involves using the Internet to provide dynamic easy extension and often virtualized resources.Cloud storage is out on the cloud computing concept extension and development of a new concept,it refers to the application through the cluster,grid technology or distributed file systems,and other functions,the network of a large number of various types of storage devices set up by applying the software work together in simple terms,the cloud storage is the storage resources in the cloud for the user to access a new plan,is a hot spot in the present study.The cloud storage system based on HDFS refers to that the end-users achieve the free access of file information and get the function of file and folder uploading/downloading and management through remote Web of Internet.This paper introduces the necessity of constructing cloud storage system based on the overall environment of IT service,and establishes the basic storage pattern with HDFS as the platform by analyzing and comparing the technical features of GFS(Google File System),Amazon s3 and Hadoop.Through setting up the Hadoop storage platform,we achieve the encoding of each sub-system by using Java Web programming technology and Mysql database,and conduct the performance test of the cloud storage system.As a third-party,this paper designs and achieves a universal campus cloud storage system based on Hadoop and studies the operating mechanism of HDFS in details,based on which we implement the overall design of the cloud storage system of a college.In the paper,we improve the original HDFS load balancing algorithm and propose a multi-index load balancing algorithm,and apply it to the data management of this system to control the distribution and migration of cluster load.In HDFS itself,on the basis of load balancing,according to the specific circumstances set evaluation index,evaluation function,design load,and double threshold strategy to determine node performance.The final load migration,completed the design of the algorithm,improve the system performance.From the analysis of system feasibility to the implementation of encoding,this paper provides a prototype system to the overall design of cloud storage system,which can be used as a reference for relevant technical personnel.The cloud storage system designed in this paper can meet the demand of data moving with users,and can also solve this college's issues in massive data storage like high cost,complex management and poor reliability,and in the meantime,builds a basec platform for their future cloud computer application.