Research And Improvement Of File Transmission Strategy In HDFS

With the rapid development of computer and network, the data quantity also developinglike surge, according to cisco in its "global cloud data report" indacated that due to the user’srequirements that unrestricted access and application of enterprise data, the global cloud datatraffic will grow at an annual rate of66%,during the period of2010to2015. The ability toprocessing data of traditional distributed high performance processing platform can’t satisfythe explosive growth of data processing the request already.Cloud computing, cloud storageemerged at the historic moment which can satisfy the request of content data intensiveservice.In this paper, the content is the research of the transmission strategy in the process oftheir reading and writing on HDFS, which is the file system in the cloud service system thatbased on HADOOP.Aim to increase the efficiency of the basic operations of writing andreading in HDFS, that to achieve a higher efficiency and highly fault-tolerant file system.Each operation in the cloud storage is inseparable from the file system calls, on the file systemcommonly used in the most basic and the most commonly used HDFS transmission process isreading and writing. if improved on that processes, we can realizing a system which can readand write in parallel, also with highly fault-tolerance, which will greatly speed up the accessof data in the cloud storage service and availability issues.Firstly, this paper introduces the related theory and technology of cloud storage,illustrates the definitions of cloud storage, and described its application occasions. Followed,it analysis the HDFS thoroughly, and give out assay and comparison of the relatedtechnology.Subsequently, it described the fault-tolerant techniques of reading and writingon HDFS in detail, which provided a good technical support for the following research. Atlast, improved the process of read/write mechanism of file transportation in HDFS after themechanism is analyzed by studying the HDFS; implement the function of file data block’sparallel transmission. That provides a reliable solution to high latency and copy securityissues in cloud storage.This paper implements the strategies of parallel transmission in the process of reading and writing in the cloud storage which based on HDFS, and strategy of copy‘s automaticreplication, that improved the efficiency of reading and writing, and reduced the delay time,which cloud provided an efficient and stable service storage to the user. In this paper,under theimproved strategy, make full use of the replicas, disperse the load of the network, the datareading efficiency can be increased by160%, the efficiency of copying replication is alsogreatly enhanced.