The Research And Implementation Of Distributed Processing Framework Based On Zookeeper

How to handle massive data effectively and get the valuable information enterprises and users need becomes an urgent problem to be solved. Distributed computing is the traditional method to deal with massive data. However, the current mainstreams of distributed framework are not designed to purse fast and flexible computing, they are more of a distributed file system, providing file data storage and query in distributed file system. Their computing tends to provide a parallel computing method for data query.The emphasis of my distributed processing framework is computing, it uses chain calculation to simplify the distributed computing, providing a more efficient method of distributed computing than Hadoop.The paper implements the computing framework and file system. The framework uses chain calculation to simplify the distributed computing, it can control the entire calculation deeply. The relationship of several steps in distributed computing is serial, the developers can set up a management node for each step, each node is responsible for managing their corresponding step, the execution inside the management node is parallel, the execution between multiple management nodes is serial, that means, the execution of next node must wait for the completion of the previous node. File system can provide necessary support for distributed computing, provide basic operation of documentation in the cluster.Experiments show that the framework can handle large mount of data efficientyly using its own high-speed file system when the data is placed in the right place. The framework can deal with the issues such as Wordcount using less time than Hadoop. But there is still disadvantage exists in the framework, it does not provide fault tolerance so it can’t handle the exception in the cluster in time.