The Design And Implementation Of The Management System Of The Hadoop Cluster

In this paper, a Hadoop cluster management system is designed and implemented. This system provides a unique browser-based interface to install and manage the Hadoop cluster, sloves the management difficulty of the open source version, and provides a system alarm mechanism with web and email. With the powerful and user friendly system, the users can visually install, configure, manage and optimize the Hadoop cluster so that to concentrate on getting the largest business value without worry about the management problem of the cluster.The main work of this paper is: 1) Expound the research background of this project, analyze the significance and application prospect of it, and summarize the related research results.2) Detailedly analyze the functional and not-functional requirements of the system, and introduce the reason of why using the technologies in the project.3) Design the overall framework of the system and explain the design of each module in detail. For example, use Akka+Spring MVC as the overall framework and use Hibernate to encapsulate the operations of the database.4) Introduce the technologies and algorithms used in the system development in detail.When the project completed, the experiment confirmed that it just took 3 minutes to install a Hadoop cluster with full components on 35 nodes in Amazon using this system. In the two years promotion of Intel Distribution for Apache Hadoop in China, it has made a number of successful applications in in the telecommunications industry, intelligent transportation industry, the financial industry and health care industry, and unanimously praised by many companies including the Agricultural Bank of China.