Design And Implementation Of Cloud Operation And Maintenance Tool For Storage-computation Separation Database

The self-developed storage-computation separation database adopts the computing and storage separation architecture.The computing node cluster has one master and multiple slaves,and only supports single write and multiple reads.The storage node cluster has multiple peer nodes,which are copies of each other,and jointly provide services for the computing nodes.At present,this kind of database has problems such as complicated deployment steps and high expenditure on node expansion.Therefore,it is of great significance to develop an operation and maintenance tool that can automate the deployment of the storage-computation separation database to the cloud,and can elastically scale database computing nodes and storage nodes according to index values and scaling algorithms on demand.Combined with the Kubernetes-related operation and maintenance technology and the characteristics of the self-developed storage-computation separation database,a cloud operation and maintenance tool scheme is designed.The scheme includes four modules: automatic deployment,resource monitoring,elastic scaling of computing nodes and elastic scaling of storage nodes.The automated deployment module uses technologies such as Docker and Harbor to design solutions for making,storing,and pulling images of computing nodes and storage nodes,and uses scripts to complete cluster initialization.The resource monitoring module uses technologies such as Prometheus to collect,store,query,and aggregate resource indicators and custom indicators of computing nodes and storage node clusters,and provide indicator data support for the elastic scaling modules of computing nodes and storage nodes.According to the dynamic scaling requirements when the access traffic of the database system changes,the solutions,including the horizontal elastic scaling of the computing node cluster based on the CPU utilization of the central processing unit,the vertical elastic scaling of the computing node cluster based on the memory utilization,the horizontal elastic scaling of the storage node cluster based on the number of queries per second(QPS)and vertical elastic scaling of storage node clusters based on CPU utilization,are designed.The computing node cluster and the storage node cluster will define their own scalers and state managers respectively,and then perform dynamic scaling according to the indicator data of the resource monitoring module and the custom scaling algorithm,which improves the availability of the system.On the basis of testing the automatic deployment and resource monitoring functions of the operation and maintenance tools,the read-only and read-write QPS performance of the database system with the scaling function enabled and the database system without the scaling function enabled is tested and compared by the Sysbench stress testing technology.The experimental results show that all functions meet the design requirements,the QPS read-only and read-write performance of the four scaling schemes are improved to a certain extent,and the resource utilization and availability of the system are improved to a certain extent.