Research And Implementation Of HDFS Distributed File System

Material,energy and information are three basic objects in the research of natural science,processing,transmission and storage are the three basic tasks of information computing.With the continuous development of network technology and information processing technology,the production of personal data and enterprise data shows explosive expansion,IT system is faced with the problem of high cost,management difficulty,low reliability.In order to make full use of resources,reduce the investment,data storage system as the main architecture and infrastructure,as a complete system from IT system,distributed storage system has been widely discussed and applied in the field because of massive data storage,high scalability,high reliability and high availability.Therefore,the research on the distributed storage system is not only the trend of the current development,but also has a high application value.Based on the research of distributed storage system,this paper aims to construct a distributed file system with high performance,high reliability,high availability,and solve the problem of storage,query,high performance and high fault tolerance.It provides high performance,high reliability and high availability for IT system.The main contents of this paper are as follows:(1)the theory of distributed architecture and the application status of distributed storage system,the concept of distributed storage system is introduced;(2)the HDFS distributed file system of the open source project Hadoop is introduced,and the key operating mechanism of HDFS is analyzed in detail;(3)and on the basis of this,the experimental environment of the experimental environment is built on the basis of HDFS.The experimental data is collected and the experimental results are analyzed and summarized;Through the combination of the actual demand,the storage system,function structure and operation environment of small and medium enterprises have been designed and planned.