Design And Implementation Of Distributed Realtime Computation System Framework Based On Strategy

The present is the time of big data.With the development and innovation of electronic technology,all kinds of digital equipment emerge one after another,which greatly improve the productivity and the quality of life.At the same time,the human's ability of digitizing the real world,and producing data also explodes.Faced with such a large amount of data,it's impossible for traditional computing machines and methods to process or store these data effectively.So,the industry try to integrate the existing computing resources and to change methods of processing data in order to achieve the purpose of processing data efficiently.This is the big data technique.Google,as the world's largest search engine company,has always been at the forefront of distributed systems and parallel computing.Its three major papers of how to process data efficiently lay the foundation for,and further opening the door toward the filed.At present,big data processing system is divided into batch processing system and real-time processing system.Representatives of open source software are Apache Hadoop batch processing framework and Apache Storm real-time computing framework,respectively.Storm,as a typical representative of real-time processing framework,has a wide range of users.This paper summarizes various technologies of system of processing big data in real time,and then analyzes and designs a distributed real-time computing framework based on user's computing strategy.This framework not only has the characteristics and advantages of the general distributed real-time computing framework,but also enable users to build computing topology flexibly and efficiently.This paper,regarding constructing computing topology flexibly and efficiently as the ultimate goal,gradually deduce the functionality of the system.The functionality includes:deploying and running the topology which built in accordance with the framework on the server;The framework can detect the running status of topology and inform the user in time,and has the ability to maintain the topology structure.It provides the primitive to modify the running topology,including modifying the topology node's functionality and the topology's structure,and managing computing resources as an assistant to deploy topology.The primitives include the ability of acquiring information of hardware and software,deleting,adding and querying information of computing resource and topology in real time.The platform can redeploy the failed topology node to the normal computing node and re-establish the braked relationship.It provides a simple and consistent framework of developing distributed program,according to which the software designer can not only develop program accordant with the framework efficiently and pleasantly,but also have a potential of further development.With the help of the platform,users can concentrate their attention on business design and computing process design.In this way,the complexity of developing business processing software is reduced and the efficiency is improved.