Research Of The Automatic Deployment Of High-concurrency Cluster Based On Container

Along with the tremendous advance of big data technologies,clusters comprising computer nodes are becoming widely prevalent in many fields,e.g.,webserver serving,business data analysis,and scientific research.Because of the increasingly size of clusters,as well as the demand of failure handling,deploying clusters in a convenient and robust way becomes more and more complicated.In order to solve the above mentioned problem,this research aims to propose,design,and implement a solution on deploying large scale clusters,as well as a highly available tool to be automatically installed on multiple platforms.This work is built upon high-reusability and cross-platform containers,with object detection as the targeted application.To complete the goal mentioned above,the practicability of research ideas was evaluated as the first step.And I find that automated deployment has good application prospects,which allows deployers to not need to pay attention to the cluster deployment process.Based on our findings,Docker and Kubernetes are selected as the main management system;Metrics Server and Prometheus are selected as custom metrics monitor component;and Etcd cluster is selected as the data storage component.Haproxy and Keepalived are integrated as the component for load balancing,to improve cluster availability.A feasibility study was then conducted to evaluate the selected components.This work proposes new solutions to provide rapid deployment,high scalability,high availability features.It is also optimized against the automatic delay response of the Kubernetes architecture at load peaks.As the core algorithm in our method,the smoothing method is combined with the gray model to predict load peaks.It then expands in advance to deal with the upcoming load peak,according to the prediction.Finally load peaks can be removed or reduced to improve application performance.The system and optimization methods are implemented into a system called Container-Based High-Concurrency Cluster Automatic Deployment System.To evaluate the implementation,experiments were conducted on the results to verify the availability of the tool,as well as the effectiveness of the optimization plan.It illustrates the effectiveness of the predictive optimization plan proposed by the study: the amount of advance time for capacity expansion can be selected to ensure the minimum container service.And a larger amount of advance based on the actual project and load period could be chosen to get more stable property by sacrificing some resources.This work makes three key contributions.(1)It integrates the container cluster and management platform into a highlyconcurrent cluster,which can be quickly and automatically deployed,whilebeing reusable,highly available,and scalable.The tool is valuable in reducingthe deployment labor cost of the cluster.(2)The auto-deployment integrates the scheme of monitoring the cluster throughcustom metrics.It also implements auto-scaling of the containers based on thecustom metrics,and supports both horizontal pod scaling and vertical podscaling.(3)The research integrates a new method of container auto-expansion foroptimizing,and change the method of container auto-expansion from aresponsive design to a predictive design,which improves the user experience tosome extent.