Research Of Mass Data Storage Technology Based On Hadoop Platform

With the development of Internet and the enlargement of Internet population, the imagedata on the Internet are growing at a dramatic pace. The image data of a company can reach ascale of TB or several hundred TB. Generally, distributed file system is used to process suchpictures.Hadoop is a recently-emerged distributed file system that can process large-scale data. Itfeatures with such qualities as Good reliability, and large storage capacity,handy deploymentand easier maintenance.Based on the above mentioned qualities, this paper is to study the image storage-relatedproperties of Hadoop and based on the Hadoop platform, this paper is to design an imagestorage system for the small and medium companies. The research content is as follows:1.The Working Principle of Hadoop Cloud Computing PlatformIn this paper, we deeply research principles of data storage and reading on Hadoopplatform from aspects of data organization, dataflow and others. In addition, we also study thework processing of MapReduce Distributed Computing Framework.2.The design of Image Storage System based on Hadoop PlatformAccording to the requirement, we design common user module, administrator module,log analysis module, client, system monitor and other function modules. Meanwhile weaccomplish the system architecture design with Hadoop, Tomcat, Mysql and other software.In accordance with function modules we also design Class Diagram of UML and Mysql tables.3.The Realization of Image Storage SystemWe build the Hadoop cluster at first, and later accomplish the realization of each functionmodule, in which we focus on operating Hadoop and log analysis.4.The Integration of Hadoop Platform and WebWe integrate Hadoop Platform and Web on the basis of Hadoop and JSP so that imagestored in Hadoop can be visited by JSP page.Finally, with this paper’s study result, we propose relevant test method to verify systemperformance and reliability.