Research On Evolutionary Based Automated Neural Network Compression

In recent years,artificial intelligence technologies represented by deep neural networks have made great breakthroughs in many fields such as computer vision,search recommendation,and speech recognition.However,with the continuous development of algorithms,the number of parameters and computational complexity of neural network models are also increasing,which seriously restricts the deployment of deep neural networks on embedded devices with limited storage space,power consumption,and battery life.The structured pruning algorithm is able to greatly reduce parameters and computation without losing the prediction accuracy of the model.At the same time,it has been widely used because it does not depend on customized hardware platforms for improving the efficiency of computing and memory access.However,there are a large number of hyperparameters which are difficult to determine in the structured pruning algorithm,and there are still many challenges in the implementation process.This paper studies the theoretical basis of the structured pruning algorithm,and on this basis,an automatic structured pruning algorithm based on heuristic algorithm is proposed.The main contents include:This paper proposes a scalable automatic structured pruning framework.Existing structured pruning algorithms have a large number of hyperparameters that are difficult to optimize,and manually determined hyperparameters are often not optimal pruning strategies.To solve this problem,this paper builds a scalable automatic structure pruning framework.The framework not only uses the hyperparameter optimization module to complete the automatic optimization of the hyperparameters in the pruning process,but also adopts a design mode in which the mechanism and strategy are separated,and supports multiple pruning algorithms.A fast assessment method of individual fitness based on equivalence assumption is proposed.The use of iterative pruning for individual fitness evaluation in the hyperparameter search process will bring a huge amount of calculation.To solve this problem,this paper proposes a rapid evaluation method based on the equivalence hypothesis,and verifies the validity of the hypothesis on multiple neural network models in the CIFAR-10 data set.An automatic structured pruning algorithm based on combinatorial optimization algorithm is proposed.The traditional heuristic algorithm is either slow to converge in the search process,or easily converges to the local optimal solution.To solve this problem,this paper proposes a combined optimization algorithm based on differential evolution and simulated annealing,which not only retains the advantages of differential evolution algorithm with fast convergence speed,but also uses simulated annealing operator to reduce the probability of converging to a local optimal solution.In this paper,based on the automatic structured pruning framework combined with the combined optimization algorithm and the rapid assessment method of fitness,a variety of neural network models are automatically structured pruned on multiple data sets,and compared with existing automatic pruning algorithms.,Verifying that the algorithm can improve the compression ratio of the model more efficiently with an automated process.