Research On Scalable Bayesian Optimization Based On Trust Region

Bayesian optimization is an efficient global optimization algorithm for expensive black box problems.It can find an ideal solution under relatively few function evaluations.Although it has been successfully applied in automatic machine learning,engineering,physics,experimental design and other fields,it also has the following drawbacks.1)The time complexity of Gaussian process inference is at leastΟ~2,represents the number of sample points.When the objective function dimension is high,it usually needs a large number of sample points to find the ideal answer,so the execution time of the algorithm can not be ignored.Although the execution time of the algorithm can be reduced by improving the computational power,it is difficult for the algorithm to perform well due to the over-explore nature of Bayesian Optimization in high-dimensional problems.2)In each iteration process,the acquisition function needs to be optimized.Usually,the function is highly non-convex and multi-modal,which is difficult to achieve good optimization effect in practical application.In order to solve the above problems,we proposed a trust region based local Bayesian Optimization TRLBO.Specifically,we use two trust regions with dynamic sizes to control the algorithm to focus more on local search,while also has the ability of global exploration.One trust region is used to limit the number of samples used to train Gaussian process in the iterative process,the trust region is a sphere with the current best observation as the center and a dynamically changing scale parameter as the radius,the data used to train Gaussian Process is the observation points located in the sphere,which can effectively reduce the number of sample points used to train Gaussian process;another is used to limit the size of the solution space for generating new sampling points,this trust region is a hyper-rectangle around the current best observation,and the next evaluation points selected in each iteration process are located in the hyper-rectangle.Finally,TRLBO uses random search to optimize the acquisition function,which avoids the cost of optimizing the acquisition function,and also make the algorithm have the ability of batch sampling.Experiments showed that TRLBO had the same or even better performance as other algorithms in the test set of synthetic function and the solution of practical application problems.