Based On HDFS Application Of The Enterprise Information System In Cloud Storage Platform

With the continuous development of science and technology, digital information is showing explosive growth. Traditional storage methods have been unable to meet the current demand for coal mass data storage, cloud storage, cloud-based systems have come into being.HDFS(Hadoop Distributed FileSystem) is Hadoop distributed file storage system, at present,many large enterprises at home and abroad to take advantage of HDFS to store and manage the vast amounts of data, HDFS beginning of the design is to store large file systems design and development, but with the HDFS storage systems increasingly wide range of applications,shortcomings and deficiencies of its existence gradually exposed, how to efficiently process and store small files become an urgent problem.This paper studies the problem of small files stored in HDFS and the development of HDFS-based enterprise cloud storage platform. First, this paper presents the architecture improvements, adding a small file processing unit in the original HDFS storage structure, aimed for small files and merge judge handling,indexing and content writing small files to append a way to merge files stored solve a large number of small files scattered storage space problems caused by waste. Secondly, in the storage structure improved,secondary index proposed mechanism, the index will be merged with the merge file simultaneously on DataNode, only one metadata records on the merged file Name Node small file name information is stored,used by level index to find a way to resolve the positioning of small files, saving NameNode memory,improve access efficiency. Finally, the development of coal enterprise cloud storage platform construction process, for example, described in detail the application of enterprise information technology platform in HDFS cloud storage platform.This article uses Hadoop 0.20.1 and Performance Test small file storage system of Eclipse as a development environment, a desk and three DataNode NameNode node node as a simulation platform improved, respectively, from the memory consumption, small file read time, small files write timing of the test, and achieved good results.