Research On Some Key Problems In Bayesian Optimization And Association Pattern Discovery

There are quantitative design problems in scientific research and industry.Hyperparameter optimization is a very important design problem in the field of machine learning,which seeks to find the optimal hyperparameter configuration from a large number of different combinations of hyperparameters to achieve the best performance.The training process of machine learning models is iterative,with shorter training time corresponding to light training measurements and longer training time corresponding to heavy training measurements.This dissertation proposes a Bayesian Truncated Additive Gaussian Process Optimization method to solve the hyperparameter optimization problem in machine learning.In the proposed method,we develop a statistical approach to jointly model the light training and heavy training measurements,and sequentially add more hyperparameter configurations with potentials to improve the performance of the machine learning model via Bayesian optimization.A series of simulation studies and machine learning experiments demonstrate that the proposed approach outperforms the existing methods.In the field of materials science,another design problem that has attracted widespread attention in scientific research and industry is metamaterial design.Composed of numerous unit cells with designable microstructures,metamaterial can present extraordinary electromagnetic properties that are not available in natural materials.This dissertation investigates the metamaterial design problem with emphasis on dealing with the challenge of designing numerous unit cells with functional responses,which is not common in traditional design problems.We formulate the multiple related design targets as a multi-target design problem.We propose an efficient synergic design strategy based on Bayesian optimization to achieve the rapid design of metamaterials.The proposed method transforms the complex functional responses into two simple responses,the mean function and the variance function,which are fitted by parsimonious surrogate models.Leveraging on the similarity between different design target,we develop an integrated acquisition function to simultaneously optimize all design targets.We also show the convergence of the proposed synergic design method.A wide range of experiments have verified the effectiveness and superiority of the proposed method,so this method has great application potentials in the field of metamaterials.With the popularity of big data applications,another important topic in scientific research and industry is discovering some latent and valuable association patterns from massive data with complex structures.Assuming that each association pattern is a theme composed of items and each observation of the data consists of several themes generated from a theme dictionary,the theme dictionary model(TDM)provides an innovative way to achieve efficient association pattern discovery via a probabilistic generative model with statistical inference.This dissertation extends the original TDM by allowing more than one category of items to appear in data and only presence/absence of items to be observed for each observation with all quantitative information missing.The extended models can resolve a larger range of practical problems that cannot be handled by the original TDM.Simulation studies confirm the superiority of the extended models.We apply the proposed extended models to analyze the publication data of journals and the electronic medical records of traditional Chinese medicine to discovery association patterns.