Evolution For Efficient CNN Search

With the development of neural network technology,researchers have achieved unprecedented accuracy on many machine learning tasks,but the amount of model parameters and computation required have increased heavily in the meantime.On the premise of maintaining accuracy as much as possible,how to reduce the model parameters and increase the inference speed has become an urgent problem to be solved in the practical application of neural network models.To improve the efficiency of neural networks,researchers have proposed many efficient neural network architecture.However,in practical application,the neural network structure for specific problems usually require expert design,which brings a lot of costs on manual trial.Neural architecture search(NAS)is an emerging technology that automatically searches for a suitable network architecture through some algorithm,which could be more effective than manual design.This paper applies neural architecture search technology to the search of efficient neural network models,and proposes a new NAS method based on genetic algorithms.A classic evolutionary method on NAS problem is the tournament algorithm.This method has the disadvantage of easily get trapped in a locally optimal solution.In this paper,inspired by the breeding method of social animals,a group tournament algorithm is proposed.The algorithm proposed in this paper is based on the tournament algorithm.By dividing the population into several sub-populations to apply the tournament algorithm,and adding the elimination and supplementary mechanism of the sub-populations,we solved the problem of tournament algorithm.In order to use NAS method to search efficient CNN,we propose several principles of the efficient neural network search space.Taking CIFAR-10 as the target,we designs the corresponding search space.We also greatly reduces the computing time required for architecture search through a reasonable weight sharing strategy.We searched efficient CNNs for CIFAR-10 dataset.The results show that compared with other NAS algorithms,the best model obtained by our search method has least parameters and competitive accuracy.At the same time,the computational resources required by this algorithm are only 7 GPU days,which is very low for evolutionary methods on NAS problem.