| Hadoop is widely used in distributed application development. Users can build a lot of high quality distributed platforms with hadoop. Hadoop system contains many components, there are a lot of related projects expanding its function, such as a ZooKeeper, Hive, Hbase, etc.Users will encounter lots of problems when using hadoop:such as how to use hadoop language for special purpose, how to invoke hadoop API, how to carry out the hadoop cluster configuration, how to write mapreduce task.This paper proposes a rapid hadoop applications development platform. The research purpose is to provide the hadoop parallel computing ability to user with reusable templates. The application system uses cloud computing technology,which has a strong scalability and reliability. According to the different needs of different users on hadoop program, customized hadoop template can be reused many times in the future. It allows users to browse the historical program, instead of performing the same repetitive program every time.The system simplifies hadoop designing and compiling process. There is no need for manual packaging and manual copying. Users only need to select a fixed template and configure it, specify the data source. After that user can successful build a program. Packaging and uploading program, data uploading, hadoop operation execution, showing the calculation results, all of this series of operations are performed by the rapid development platform.The paper firstly introduces the concept of hadoop and parallel computing concepts.lt also introduces the development of the hadoop, as well as the background and significance of rapid hadoop applications development platform. Then, several key technologies are introduced. After solving the key technical problems, this article proposes how system interacts and the data processing procedure between each function module. Finally, the system function test and performance analysis are proposed.Also putting forward the improvement and works in the future. |
PDF Full Text Request