Evolutionary Algorithm-Based Neural Architecture Search

Automatically searched architectures by neural architecture search(NAS)methods have shown promising performance in various visual recognition tasks.Among NAS methods,evolutionary neural architecture methods are popular because of their potential to find the global optimal convolutional neural networks(CNNs).These methods usually use an individual to represent an entire CNN,while often facing three challenges: 1)Since a CNN has numerous encoded parameters and weights,the length of each individual is long which causes a large search space.2)Due to the unknown optimal depth of a CNN,it is necessary to deal with a variable-length optimization problem,which leads to the search in a messy way since search spaces with different dimensions may have different optimal solutions.3)During evolution,multiple parts of one individual will changed at once because of the crossover and mutation algorithm,which makes the search process unstable.In this paper,we propose a genetic algorithm with a simple encoding scheme for evolving CNNs(Layer-Based)to address image classification problems,called SEECNN-L.In our encoding scheme,the parameters and weights of each layer are encoded into an individual and the whole population represents an entire CNN.Over the course of evolution,three offspring subpopulations are separately produced by genetic operators on three subpopulations.In each subpopulation,the lengths of individuals are short and equal.Afterward,we design a stable search strategy to update the population based on the performance improvement,where we only insert,replace,and remove one individual to generate candidate populations.Besides,based on SEECNN-L,we design SEECNN-B to search BlockBased CNNs in order to achieve higher performance.By using the block structure as the basic unit in search process,the classification accuracy of the searched CNN is further improved.In order to verify the effectiveness of the algorithm,this paper compares SEECNN-L with 14 algorithms on 9 public datasets,and compares SEECNN-B with 15 algorithms on 5 public datasets.The results show the effectiveness of above two algorithms.Furthermore,on the realworld skin disease classification dataset,the two proposed algorithms also exhibit better performance than human-designed architectures.