Supervised AutoML Performance Optimization Based On The Selection Of Key Hyperparameters

Recently,the research and application of machine learning have achieved fruitful achievements in various fields.These achievements mainly rely on the manual intervention of machine learning experts in various aspects,especially on the algorithm selection and hyperparameter tuning stage.The advent of Automated Machine Learning(AutoML)has reduced the burden on experts and transfers the focus of work from tedious and repetitive tasks to data analysis.As we all know,people usually improve the performance of model by tuning the settings of the hyperparameters.This process is called hyperparameter tuning,which is the most time-consuming stage of AutoML.In this process,because the difference between model performances is vary from hyperparameters,the selection of key hyperparameters has significant application value and research significance for improving the performance of AutoML.This paper is oriented to the combination optimization problem of algorithm selection and hyperparameter optimization(which also called CASH problem)in traditional supervised algorithms,focusing on the intrinsic relationship between hyperparameters and model performance,and deeply researching the search strategy in the process of AutoML configurator.For its performance bottlenecks,optimizing it by a new component which based on key hyperparameter selection techniques.The main research contents of this thesis includes:(1)analyzing the performance bottleneck of the excellent supervised AutoML configurator SMAC,and found that in the configuration generation stage,the acquisition function of SMAC configurator generates additional unnecessary evaluation due to low performance configuration with high uncertainty.(2)Analyzing the intrinsic relationship between hyperparameters and model performance,using Mean Decrease Impurity(MDI)method to quantify the contribution of hyperparameters to model performance,and proposing a pruning strategy based on key hyperparameter selection to construct a new Pruning component.(3)Based on the SMAC configurator,the new configurator Pruning-SMAC is designed and implemented.This new configurator has the dynamically adjustable search space,and includes new Pruning component proposed in this paper.It can collect historical performance data during the iteration process of SMAC to prune range of hyperparameter,thereby gradually obtaining the core search space and avoiding to select configuration in the area with poor known performance,reducing unnecessary overhead,improving the AutoML performance.