A Multi-View Learning Based Auto-Scaling System In Public Clouds

Current cloud computing mainly adopts the pay-as-you-go model,which requires tenants to estimate the amount of rented resources(i.e.,virtual machines)based on the service requirements(e.g.,deadline)of the computing job(e.g.,MapReduce).However,the performance of virtual machines in public clouds is usually unstable,which makes it hard to guarantee the job finished by deadline.In order to ensure the on-time completion of the job and reduce the cost of rented resources,tenants need an automatic resource management system to adjust the amount of resources in real time according to the running status of the job.A multi-view learning based auto-scaling system can be used to solve the above problems.This system includes an initial resource recommendation module and an elastic resource management module.The first module will provide the initial resource recommendation of the most cost-saving amount of resources for the tenant based on the deadline and workload.During the execution of the job,the multi-view learning based neural network of second module can extract features from three views of information(i.e.,computing resources,resource utilization and running speed of the job),and make more accurate predictions about the completion time.This model is able to capture the variance in the job's running speed and timely adjust the predictions when the performance of the virtual machine degrades.When it predicts that the completion time of the job will exceed the deadline,this module will increase the number of virtual machines,which increases the running speed of the job.In Alibaba Cloud,this system has been implemented based on Hadoop.Experiments show that: 1)the prediction accuracy of the system achieves 98.4% under the condition of determined initial amount of resources;2)compared with other existing resource management systems,this system can save up to 30.8% of renting cost for tenants while guaranteeing the job finished on time.