Design And Implementation Of Integrated Deployment Platform For Distributed Business Systems

With the coming of big data era and diversified development of application,academic circle and industry circle designed and delivered multiple distributed business systems as programming framework in order to address the challenge of the increasing data type and demand of application, such as off-line batch processing framework Map Reduce、stream computing framework Storm、memory iterative computation framework Spark、DAG computation framework Tez、schemata computation framework Pregel. In the foreseeable future, new computation framework probably appears. Every framework focuses on specific application scenario. It’s unlikely that one computation framework can be the optimal option for all application scenarios. Distributed business system is always closely connected with general services in production environment,such as Web server、database server. There will be some issues, like poor scalability, low utilization rate, if we offer dedicated infrastructure and individual design for each distributed business system.To solve above issues, we analyze the structure and characterization of current deployment support platform and design Distributed business system Deploy Platform(DDP) based on existing research status, offer distributed business system with automated deployment and basic support service of highly scalable customization. The main job and creativity is as below:(1) Achieve the function of cluster multiplexing under which we are able to run multiply calculation framework or application. The task is assigned to the properest load node according to its heterogeneous resource requirements. Dynamic expansion and contraction based on business load is realized. Modules obtain computational resources according to the demand.When the load exceeds a high threshold value,expand business modules to allocate more resources. And shrink them when the load is less than the low value to recycle idle resources.Design the mechanism of monitor, which can monitor status of business system model. It can transfer request to other deployed nodes when certain models are failed, this can guarantee the serviceability of system.(2) Optimize the dual scheduling model, and resources is distributed with multiply scheduler under share status, increase concurrency.(3) Adopt bearer service system with open source container, improve isolationperformance and scalability.The outcome indicates that this system is capable of monitoring working status and with the ability of fault-tolerant processing after the test of function. Also, it can distribute the resources reasonably.