Convolutional Neural Network Pruning Based On Evolutionary Algorithm

Recent years,as the development of the deep convolutional neural network,neural network pruning and compression attracts more and more attention in the academic and industrial circle.Previous pruning methods mainly leverage hand-crafted criteria that require a large amount of human heuristics and expertise for proper ad-hoc criteria.Therefore,this thesis studies AutoML pruning methods to automatically prune channels and proposes two AutoML pruning algorithms based on Evolutionary Algorithm.First,this thesis proposes an AutoML pruning algorithm based on Evolutionary Algorithm,called MetaSelection.MetaSelection reformulates the neural network pruning as a combinatorial optimization problem from the view of set selection problem and exploits EA to search the proper sub-structure satisfying the resource constraints.Regarding the tremendous search space of channel selection,MetaSelection further utilizes a coarse-to-fine pruning strategy to reduce the search space.Moreover,a more effective crossover strategy based on probability distribution is used to further accelerate the evolutionary search.Due to the search space and the reliability of pruned network evaluating problem in MetaSelection,this thesis then proposes Evolutionary Pruning via Slimmable Network,EPSN,which introduces Slimmable Neural Network into the pre-training process to further reduce the search space of evolutionary pruning search and improve the reliability of pruned network evaluating.Slimmable neural network uses several width factors to discretize the continuous width search space and allows EA to directly search for proper pruned network structure which greatly reduces the search space.Moreover,original slimmable neural network can only use uniform width factor for all layers,this thesis uses a modified sampling and training strategy to allow slimmable neural network to evaluate all potential width combination.Experiments of pruning various appealing deep neural networks on several datasets demonstrate the effectiveness and superiority of the proposed algorithms.Compared with other AutoML pruning methods and criteria based pruning methods,the proposed algorithms achieve comparable or superior Top-1 accuracy under the same constraint of model size.