Research And Application Of LSTM Based E-Commerce Platform Kubernetes Cluster Elastic Scaling

With the rapid development of the Internet industry and the great enrichment of material life,a large number of Internet e-commerce platforms have emerged at home and abroad.The continuous expansion of data volume and increasingly complex business scenarios put forward a major test to the performance and stability of the infrastructure supporting the e-commerce platform.It is important for managers to control operation cost scientifically while ensuring system performance and stability.This paper based on the service and cloud native structure of micro Internet business characteristics of the electric business platform and the load of Kubernetes cluster index analysis,points out the problems of waste of resources and services jitter:load index change law is closely related to the characteristics of human life,there is an obvious and cyclical volatility,at low load will waste a lot of cluster resources.Although passive elastic strategies such as Kubernetes cluster Pod horizontal automatic Scaling(HPA)can save some resources,it takes tens of seconds for new instances to be scheduled and finally available during capacity expansion.During this period,service jitter may occur,resulting in service quality deterioration.In view of the above problems,this paper proposes a Kubernetes active elastic scaling scheme based on Long-Short Term Memory(LSTM)network.On the one hand,LSTM is used to proactively predict and schedule the Kubernetes cluster load.On the other hand,Kubernetes cluster with shared GPU support is also used as a carrier to predict application deployment.The specific research contents are as follows:1.Studied the principle of Kubernetes cluster Pod horizontal automatic expansion and contraction,and analyzed the reasons of service jitter and quality decline caused by passive elastic expansion and contraction scheme.An active elastic expansion scheme is proposed to expand the number of service copies(Pod)in advance in anticipation of the peak load.Combined with the characteristics of cluster load data and business scenarios of e-commerce platform,common time series prediction models(ARMA based on linear model,decision tree model based on ensemble learning,LSTM model based on recurrent neural network)are studied,analyzed and compared.LSTM is the most suitable model for load index prediction of e-commerce platform,which improves the jitter phenomenon during service expansion and improves service quality.2.Build a service load index prediction model based on LSTM.Firstly,the load indexes were collected and preprocessed.Then,regularization,epoch adjustment and random inactivation were used to solve the over-fitting problem,and adjusted excitation function and Batch Norm method were used to solve the problem of gradient disappearance.The network depth,number of neurons,optimizer and learning rate were repeatedly adjusted to optimize the model.In order to improve the LSTM model training speed,the principle of model training on GPU and Kubernetes cluster GPU sharing scheme are studied,and a Kubernetes cluster GPU sharing scheme based on Gaia GPU is proposed.Finally,the problems of GPU program resource sharing and resource isolation in Kubernetes cluster are solved,and the training speed and prediction accuracy of LSTM prediction model are improved.3.Based on the above research content,a Kubernetes cluster Pod predictive level expansion component-PHPA is designed and implemented.The component follows the principle of cloud native design and deployment,and is highly adaptable to Kubernetes.Through simple configuration,it can automatically collect load indicators,train prediction models,forecast load indicators and schedule service copies for the specified services in Kubernetes cluster.After testing,PHPA component can better adapt to the load characteristics of e-commerce platform,effectively solve the problem of service quality decline caused by the lag of traditional responsive elastic expansion and contraction,and significantly improve the quality of service.