Performance Evaluation Of Cloud Platform Based On Spark

It is crucial to evaluate performance of a cloud platform and determine the main factors influencing the property.Moreover,the analysis results of related performance indicators can be applied to making theoretical predictions about the performance status of the cloud platform.This thesis studies performance evaluation of cloud platform based on Spark,which mainly includes the following aspects.Firstly,in order to solve the problem of the interrelations between the performance indicators based on the Spark technology of the cloud platform and the load performance of the cluster,this thesis proposes the analytic frameworks of Spark performance analysis,the specific indicators analysis as well as the prediction models towards the cluster load.In the execution of batch processing applications,Spark clusters can use multiple linear regression technology to calculate the correlation between the actual performance index data and the cluster load performance,as well as the index weight values,and determine the main index factors that affect the performance of the load.Secondly,there are many problems in the cloud platform,such as the difference in computing power,the difference in the size of the node processing data,and the uncertainty in the execution of the program,which make the cloud platform very different in the execution time of each task.In order to enhance the accuracy in load execution time prediction,and reasonably guide the user to apply for Spark cluster resources,this thesis puts forward the time index fusion calculation scheme and a Standard Regression Coefficient-based Weighted Support Vector Regression time prediction model(SRC-WSVR).The experiment results show that the prediction model proposed can provide users with effective data reference for predicting Spark resource cost.Finally,the prototype system is designed for both previously mentioned algorithms.Index analysis algorithm can effectively and objectively calculate the index weight,and the prediction algorithm can accurately estimate the execution time of the platform and provide a favorable reference for the leasing of cloud platform resources.