Design And Implementation Of Distributed Data Storage Based On Hadoop

Today, with rapid development of the Internet, all sorts of application of cloud computing can be heard everywhere. So many cloud apps seem the coming of age of real cloud. Therefore, the products based on the cloud computing become the most popular product of the cloud era. In addition, since the conventional Web applications save data to the database system, traditional web project is faced with enormous challenges with the increase in the amount of user data. Moreover, maintenance costs of server are very high. Data being saved directly to the cloud server will not only guarantee the security of data, and provide plenty of storage space, but also greatly reduce maintenance costs.The paper mainly design and implement a cloud storage application. In Hadoop cloud environment, the Web server program is developed to achieve cloud storage applications by operating Hadoop Distributed File System API. System uses the Struts2, hibernate3 and spring3 framework to develop J2 EE MVC three-tier architecture applications, using log4 j configuration and standardize the console output system log information, XFire development WebService to provide the corresponding service, Apache Mail realization of e-mail sending, using jQuery to achieve the page without refresh operation. MySQL is adopted to manage user data information in order to avoid file loops HDFS traverse considering the small amount of user information data in this application.From the perspective of system functions, a cloud storage system is realized. The users can access, manage, upload, download, delete and share their own data files via a web browser anytime and anywhere and so on. Users can manage own basic information at the same time.The system is distributed system architecture and different from the common web application. After testing, the access speed, response speed is obviously faster than that of the common web application. Not only that, the other machines will play this role for file management when a server is bad due to the Hadoop HDFS automatically backup data file, stored in the different cluster environ-ment from the slave server.