Research On Optimizing Deep Convolutional Neural Networks By Swarm Intelligence

In the field of deep learning,deep convolutional neural networks(DCNN)are the most widely used type of neural network,and have shown outstanding performance in tasks such as image classification,object detection,and natural language processing.The optimization problem of DCNNs is also one of the core problems in the field of deep learning.Choosing appropriate optimizer parameters and network structure is crucial to achieving good model performance.On the one hand,as the depth of the network increases,local optima and saddle points appear more frequently,affecting the performance and efficiency of optimization algorithms.On the other hand,network structure design is becoming more complex,and even if designers have expertise in deep learning and task objectives,they still need to spend a lot of resources and time to design networks with good performance.This makes DCNN parameter and structure search optimization a hot research topic.Due to its advantages such as global optimization and strong robustness,ant colony optimization algorithms have been widely used in many fields such as path planning,power systems,intelligent transportation,and network security.Some exploratory work has also been done in the optimization of DCNN parameters and structures using swarm intelligence optimization algorithms,and research in this field is ongoing.Therefore,this paper proposes to optimize DCNNs based on ant colony optimization.The main work is summarized as follows:(1)In terms of DCNN parameter optimization,in response to the problem that the classical stochastic gradient descent algorithm is prone to local optima,saddle points,or oscillation,and the existing swarm intelligence DCNN optimizer has a huge computational cost,this paper uses the gradient information of the gradient descent algorithm as guidance,and combines it with the idea of continuous domain ant colony optimization to propose a new ACO_R-SGD optimizer.The latest batch of training is used as a sample,and the ACO_R algorithm is used to generate gradients for optimization at a certain frequency and probability.At the same time,this mechanism is extended to more advanced Momentum,RMSprop and Adam optimizers.The four ant-based optimizer and their original versions are used for image classification training.Experimental results show that in most cases,the accuracy of the test set using the ant colony algorithm-based optimizer is better than the corresponding original version,and the training results are more stable.(2)In terms of structural optimization of DCNNs,efforts have been made to address the issues of high computational complexity and suboptimal performance in existing neural network structure search algorithms based on evolutionary algorithms.This paper proposes an ACO_R-CNN algorithm based on ant colony optimization for neural network structure search.Firstly,ACO_R-CNN proposes a search strategy based on ACO_R,which takes the entire population and its corresponding network structures as learning objects,and the layers of more excellent individuals’networks are more likely to be retained.Secondly,ACO_R-CNN allows searching for the best structure in populations of different depths.Moreover,it independently updates the individuals of fully connected layers to ensure the network’s effectiveness.Finally,it uses an epsilon-greedy strategy to generate individuals,increases population diversity,and avoids the population structure becoming too homogeneous in the later stages of the search.This article uses ACO_R-CNN and multiple similar algorithms to conduct experiments on five datasets.The experimental results show that ACO_R-CNN outperforms similar algorithms on two datasets and still has considerable competitiveness on other datasets.