Based On The Hadoop Technology Application Research On Distributed Data Storage

As countries to speed up the pace of information technology, the huge amounts of data information, really need a kind of effective method for safety management and use of the huge amounts of data efficiently, therefore, the development of the cloud storage to get fast, cloud storage research is also various. Includes a large file storage, also including the system scalability, reliability, and speed, etc., in these research topics, driven by data storage system is also developed from centralized to distributed cloud storage. Hadoop is the core technology of cloud storage, so distributed data storage in cloud storage based on Hadoop plays a backbone role.This article first study of Hadoop architecture and related technologies, including the HDFS and graphs, the key techniques such as the detailed study of the characteristics and architecture of HDFS, and use it to load balance mechanism to improve the efficiency of the system; Through the research on the key technology of distributed cloud storage for later, began to:design of distributed storage system, the main content of the system design includes:first, the system functional requirements and performance requirements are studied, made the system design principles and design goals. Secondly, through the system client and server in the research on how to ensure the safety of communication to communicate, decided to use HTTPS with SSL and digital certificate to ensure the safety of network transmission and certification. This system is to establish the Hadoop cluster under Linux system, under the laboratory environment simulation implementation of data access system. In this paper based on the Hadoop distributed cloud storage solution will effectively support the PB level data storage, the usability system support, support efficient statistical analysis.Based on the Hadoop distributed data storage technology study, realized the data according to the list of distributed storage, in the column of the query and task decomposition and set on the basis of the implementation of the query function, achieve the expected goals, has a certain application value.