Research And Application Of Convolution Neural Networks Hyperparameters Optimization Method Based On Improved Particle Swarm Optimization

Convolutional neural networks are widely used in various fields.However,designing a good network structure has always been difficult.Hyperparameters determine the structure of convolutional neural networks.These hyperparameters cannot be trained and need to be manually set.Due to the variety of hyperparameters combinations and the range of values,existed methods rely on experiences.They require a large number of manual trial and error to find the appropriate values.Therefore,how to design an efficient method that can automatically select hyperparameters is very challenging and meaningful.Although there are some optimization algorithms for hyperparameters,they do not address the above problems for both structural hyperparameters and numerical hyperparameters,and need a lot of computing resources and time costs.Based on the improved particle swarm optimization algorithm,considering the characteristics of hyperparameters,this thesis study the optimization of hyperparameters in CNNs.The proposed algorithm includes a particle coding method for hyperparameter features,an improved method for particle search ability and time efficiency,and optimization method of convolutional neural network hyperparameters.Around these three parts,firstly,according to the characteristics of different kinds of hyperparameters,the corresponding coding schemes are proposed:for structural hyperparameters,the particles are required to express the structural information of the blocks and ensure the diversity of blocks;for numerical hyperparameters,the particles encode a type of hyperparameters with each dimension of particles,ensuring multiple combinations of different hyperparameters.Then,for the premature defects of the particle swarm optimization,the learning factor of the range-adaptive adaptation and the worst particle replacement algorithm based on the cloud model improve the search ability of the particles;the particle prediction mechanism based on the least square method is proposed to address the problem that fitness calculation costs too much.The algorithm can not only optimize the existed convolutional neural network structure,but also automatically design a new network structure.To analyze the improved methods proposed in this thesis,this paper perform some controlled variable experiments.Then this paper perform the algorithm on another typical convolutional neural networks.After the optimization of the algorithm proposed in this thesis,the accuracy increases by 13.41%,which is 5.38%higher than the current best optimization algorithm.After that,this thesis uses the proposed algorithm to automatically design a convolutional neural network structure,and compares it with the currently widely used CNN structure optimization algorithms MetaQNN and NAS on several commonly used image classification datasets.The algorithm in this thesis is 1.52%higher than MetaQNN.Although it is 1.4%lower than NAS v3,the optimization time and usage resources are reduced by 4 times and 100 times,respectively.Experiments show that the proposed algorithm not only outperforms similar algorithms in terms of accuracy,but also greatly reduces the optimization cost.