Study On Hierarchical Modular Representations For Differentiable Architecture Search

Deep learning has shown great advantages in complex tasks such as image recognition,language modeling and machine translation.However,the success of these tasks is attributed to the hand-crafted specific architecture for a particular task.Generating a hand-crafted architecture is a time-consuming process,which not only needs trial and error,but also requires designers to have a high level of professional knowledge and design experience.Therefore,more and more researchers begin to explore methods for neural architecture search.In this area,most researchers work as follows: First,the search space is defined to limit the network architecture range that the algorithm can search;then,the network architecture is obtained by sampling network architectures in the search space through reinforcement learning,neuroevolution and other search methods;the accuracy of the network is obtained by a performance evaluation strategy and provide feed-back to the search method.Finally,the optimal network architecture is obtained through continuous iteration of sampling and evaluation.Among the many neural architecture search methods,differentiable neural architecture search is favored by more experts and scholars because of its high efficiency.However,most of the researches based on this are the optimization of the algorithm or the improvement of the performance evaluation strategy,which ignore the exploration of the search space.In view of this,this paper proposes a method of implementing the hierarchical modular representation in the neural architecture search to further explore the search space.The proposed neural architecture search method is called Hierarchical Modular Representations For Differentiable Architecture Search(HMR-DARTS).The algorithm adopts 8 general operations as the initial operation,and the P_DARTS algorithm is used to obtain the first-level module in the initial operation,which is then used as the initial operation of the second-level search to obtain a more complex second-level module,and finally uses the second-level module to form the final network architecture.In order to verify the effectiveness of the algorithm,we used the benchmark CIFAR-10 dataset for image classification and compared the network performance at different levels.At the same time,different types of neural architecture search algorithm are also used in the comparison.The experimental results show that HMR-DARTS algorithm is more competitive than the current most advanced neural architecture search algorithms.Compared with P_DARTS algorithm,the accuracy of CIFAR-10 dataset was improved by 0.06% on average.