Research And Implementation Of Black Box Tuning Based On Bayesian Optimization And GAN Model

The black-box function optimization problem with high cost of running has attracted more and more attention in academia and industry.Black-box optimization is widely used in life and production.Many practical systems are equipped with adjustable parameters.As the environment changes and needs fine-tuning,parameter adjustment is the most convenient and economical method to optimize performance,maximizing system efficiency and increasing economic benefits.However,black-box system is too complex to obtain any prior knowledge,let alone to model,thus we often run systems to evaluate the performance of a given input to tune parameters in actual situations.Such evaluation usually consumes a lot of time and resources.Therefore,the practical black-box function optimization problem usually has a strict limit on the number of running black-box systems,thus a relatively sufficient search cannot be performed.Moreover,the characteristics of the performance curves of different black-box functions are different,and their continuity and extreme value distribution are full of uncertainties.At the same time,the parameters and types of actual problems are different,making the parameter space huge and complicated.All of these have brought great difficulties to the search of the final better parameters,and the performance of many previous work has been greatly reduced.To solve the above problems,the text firstly standardize the definition of the black-box optimization problem,and use an optimization method based on machine learning.Finally we proposed a novel Solution by combining Bayesian Optimization and Generative Adversarial Networks.The specific research content includes the following aspects:(1)It is known that Bayesian Optimization has a good convergence guarantee,but Bayesian Optimization may take too much time to explore complex spaces,so its performance is not always satisfactory under a limited number of function evaluations.On the other hand,the Generative Adversarial Networks method has been proven to have the ability to learn the hidden structure of the existing better solutions and generate a set of possible better solutions.Therefore,this paper proposes the BOGAN method.By firstly sampling the better points with Bayesian optimization,then learning and recommending using generative adversarial networks,the designed method has a strong ability in exploration and exploitation.(2)This paper implements the BOGAN algorithm and uses three different types of experiments to verify it,including five high-dimensional benchmark test functions,three machine learning algorithms and eight software systems deployed in the cloud,which try to include black-box problem in as many fields as possible.The optimization results are also measured from the function value and the overall distribution,which not only guarantees the optimization ability,but also takes the average quality of solution into account.The experimental results show that BOGAN has better robustness and generalization,which are always better than the five state-of-the-art optimization algorithms,with an improvement of29%-74%.(3)This article also elaborates on the algorithm itself.In terms of hyperparameters,the significance of these parameters and the rationality of their values are discussed in detail according to the core idea of optimization.Starting from the loss function of the algorithm,various performances of the algorithm are analyzed.At last,we also explain the optimization principle of BOGAN in detail.In summary,this article proposes an effective method for the costly black-box function optimization problem.This method has strong generalization that can tune a variety of different black-box systems with limited number of evaluations,and search for configurations with the best performance as possible.Finally,we conduct sufficient experiments to verify the effectiveness of algorithm from two aspects.