Research On Back-Box Optimization Algorithms Based On Density Estimation Neural Networks

With human society and technology’s continuous development and progress,optimization problems in various fields such as production,life,and scientific research are becoming increasingly prominent.As the fields expand,the types and difficulties of optimization problems also increase.However,the internal principles of these problems often cannot be represented by explicit mathematical functions and are therefore known as “black box problems.” For example,the relationship between cement’s raw material ratio and cement’s strength is a typical black box problem.The raw material ratio in cement production has a crucial impact on product quality,energy consumption,and environmental protection.Optimizing the raw material ratio of cement can effectively improve product quality,reduce production costs,and reduce environmental pollution.Black box optimization algorithms are one of the methods for solving black-box optimization problems,such as optimizing cement raw material ratios.These algorithms can optimize based solely on the correspondence between inputs and outputs in uncertain mathematical problems.To address black-box optimization problems arising from various real-world scenarios,researchers have proposed many black-box optimization methods over the past few decades.These optimization algorithms are designed for different black-box optimization problems and can only solve specific characteristics of such problems.However,the black-box nature of optimization problems limits our knowledge of their characteristics,making it challenging to select suitable optimization algorithms.Ideally,an algorithm that can achieve optimal results across all black-box optimization problems would be desirable.Unfortunately,the “No Free Lunch Theorem” has proven that the optimization performance of an algorithm is highly dependent on the problem being optimized.The best optimization results can only be achieved when the prior assumptions of the optimization algorithm match the optimization problem.This means that no single black-box optimization algorithm can perform optimally on all problems.Within the constraints of the “No Free Lunch Theorem,” a feasible solution is to construct an algorithm that can achieve acceptable optimization results on most problems.Density estimation methods based on neural networks can accurately establish arbitrary probability distributions.Leveraging the strong fitting ability of density estimation neural networks,we can extract prior knowledge of the problem during optimization,enabling us to construct a weak prior general-purpose black-box optimization algorithm.In addition,existing black box testing problems are either based on mathematical functions,which are difficult to compare with realworld problems in terms of complexity,or they use a limited number of real-world optimization tasks,making it difficult to evaluate optimization algorithms comprehensively.Therefore,this thesis explores and researches the application of density estimation networks in the fields of optimization algorithms and testing problems;the main research contents of this thesis include the following aspects:(1)A mixed density estimation algorithm based on generative adversarial networks is proposed,which uses an improved generative adversarial network to model the mixture probability distribution.Compared to other algorithms,it can adapt to more complex problem structures,enabling it to achieve comparable results on most optimization problems.The algorithm introduces a mixture density estimation method to dynamically balance the exploration and exploitation ability of the model,ensuring performance.(2)A black box optimization algorithm is proposed based on deconvolution density networks,which use deconvolution density networks to establish a probability model.As an explicit density estimation method,deconvolution density networks can be trained using the maximum likelihood estimation method,allowing them to establish more accurate probability models than generative adversarial networks.The algorithm introduces a concentration mechanism to solve the problem of modeling minimum precision and a particular historical information construction method to adjust the balance between exploration and exploitation,ensuring the effectiveness of finding the optimal solution.(3)A black-box benchmark problem construction method based on density conversion is proposed,which uses a density conversion neural network to transform the input of black-box problems into a probability distribution.Without modifying the original problem,the method transforms the problem to increase its complexity,thereby ensuring better testing of the optimization performance of black-box optimization algorithms.