Research On Hyper-parametric Optimization Based On Improved Particle Swarm Optimization Algorithm

With the rapid development of cloud computing and mobile internet,the era of big data is coming.Data is very important for any industry.On the basis of data determining the research limit,how to use the algorithm to approximate the upper limit efficiently is the concern of all researchers.Machine learning method can obtain potential information from massive data quickly and effectively.In this respect,deep learning method is outstanding,and it shows strong computing power in the face of massive data.Before using machine learning algorithm,we need to configure the hyper-parameters of the model in advance.On the same issue,the performance of models with different hyper-parameters configuration is often quite different.Before the emergence of large machine learning algorithms,people mostly relied on experience to manually adjust the model's hyper-parameters,but with the increasing complexity of the model,this method obviously can not meet the needs.Based on this,this thesis focuses on the problem of hyperparametric optimization,and proposes a hyper-parameter optimization method based on improved particle swarm optimization algorithm.The main works of this thesis is as follows:(1)A hyper-parametric optimization algorithm based on discrete binary particle swarm optimization(BPSO)is proposed.It simulates the foraging behavior of birds in nature and searches for the optimal solution according to heuristic search idea.The classical particle swarm optimization algorithm is suitable for continuous space optimization problems,while the hyperparameters of machine learning models are mostly discrete variables.In this thesis,a BPSO algorithm is proposed based on the characteristics of hyperparameters attributes.In BPSO algorithm,the position of each particle is encoded by binary coding,and hyperparameters are searched by combining particle velocity information.Experiments show that this method is superior to genetic algorithm and random search method for four benchmark functions,neural network and random forest model.(2)A hyper-parametric optimization method based on memetic algorithm(MA)is proposed.Considering the above BPSO algorithm may fall into local optimum when optimizing hyperparametric parameters,a MA algorithm is proposed.Comparing with the basic evolutionary method,the memetic algorithm adds the local search stategy in the evolutionary process,which improves the search ability of the algorithm and effectively avoids the solution falling into the local optimal problem.The local search method in MA algorithm uses the BPSO method proposed above.In order to validate the effectiveness of the algorithm,the proposed algorithm is compared with the previous hyper-parametric optimization methods on four test functions and MINIST handwritten data sets.Experiments proves that the method presented in this chapter performs better and the model is the most stable,regardless of the convergence speed or the optimal value.(3)According to the existing Synthetic Aperture Radar(SAR)image classification tasks rely on manual parameter adjustment,the idea of hyper-parametric is applied to SAR image classification,and a hyper-parametric optimization method based on quantum particle swarm optimization is proposed.The algorithm optimizes the hyper-parameters of the convolutional neural network which is used to solve SAR image classification tasks,and evaluates the algorithm performance through classification accuracy.The experiment shows that the performance of convolution network optimized by manual parameter adjustment and hyper-parameter is quite different in SAR image classification.The generalization ability of convolution network optimized by hyper-parameter is stronger,and the accuracy of SAR image classification is higher.