Research And Application Of Search Space Optimization For Convolutional Neural Network Structure Search

Convolutional neural networks occupy an important position in the areas of modern computer vision.In recent years,various forms of the human-designed architecture of convolutional neural network have continuously exceeded previous records both in performance and speed,and recently have surpassed human levels in many visual tasks.The artificial neural network is more and more mature,but its development meet bottleneck.Lots of researchers have turned their attention to automatic neural architecture search(denoted as NAS),especially convolutional neural network architecture search.NAS is divided into three parts:search space,search algorithm and model evaluation,each of which worth further research.The search space directly determines the upper and lower of the searched model performance,and the search algorithm affects the accuracy of best model found and time costs.Whether the way of model evaluating is efficient determines the time and hardware cost and its impact on entire search time cannot be underestimated.Aiming at the current situation that NAS work has inadequate exploration on the search space,this paper focus on designing experiments of primitive operation set selected and the backbone network applied in the search space.Based on the search space of NAS and man-designed convolutional neural network,this paper selects a primitive operation set according to certain rules and add two extra operations,which popular in modern artificial CNN,to generate a new search space.Experiments show that under the competition mechanism DARTS(original DARTS)search algorithm,these two primitive operations are very competitive on each topology edge with others.Which means the network architecture inferred tends to choose these two primitive operations.But if we use the same of intermediate nodes as DARTS,its network performance is worse than DARTS,which indicates that the competitive DARTS search algorithm has a bias when searching in the search space.This paper introduces a cooperative mechanism to let final architecture inferred reasonably use the number of every primitive operations.It proves that the resulting FairCNAS network performance is significantly better than the original DARTS.Inspired by Huffman coding,this paper proposes a huffman-like backbone network.Experiments show that the search space constructed by the backbone network can better evaluate the search algorithm and it allows introduce existing differentiable search algorithm.The experiment proves that the network searched under the Huffman backbone network is better than DARTS backbone network.Finally,this paper applies NAS to image segmentation task and introduces the previous improvement schemes in the search space.Based on that propose NASUnet and HM-Seg network,which perform better than existing benchmark models in medical image segmentation and natural image segmentation respectively.