Research On Cell Structure Sharing Based On Differentiable Architecture Search

Image classification is a fundamental task in the research of computer vision.With the development of deep learning,neural network is used in image classification tasks more and more.It is used to find a suitable neural network architecture on certain dataset by manually designing in the past which is time-consuming and compute-intensive task.The emergence of Neural Architecture Search(NAS)makes it possible to search automatically for the most suitable architecture on the dataset,while the search space is usually very large and difficult to compute.The excessive computational overhead problem of traditional NAS is solved until Differentiable Neural Architecture Search(DARTS)is proposed,DARTS has become a popular research topic in NAS.However,there are still many drawbacks in DARTS,the search space is cell-based and stacked by same cell,which reduces the number of model weight parameters and search time,but affects the variety of the search space greatly and the accuracy of the architecture.DARTS also has a problem of network parameters discontinuity between search stage and evaluation stage,which affects the architecture performance.In order to solve the problems existing in DARTS,improve the performance of architecture and reduce the search time,we proposed Differentiable Architecture Search with Distinct Normal Cells(DNC-DARTS).The research of this thesis is conducted as the following:(1)This thesis studies weights sharing between cells,the single cell-based search space in DNC-DARTS is improved by breaking the traditional architecture stacked rule.A new search strategy is also proposed that divides the search stage into two stages,which determine the topology of architecture firstly and search the operation of architecture secondly.(2)To address the problem of network parameter discontinuity between search stage and evaluation stage in DARTS,depth and width are increased progressively in the two stages of search stage in DNC-DARTS,and the accuracy of architecture is improved by reducing the network configuration gap between search stage and evaluation stage.(3)The problem of skip connection aggregation is solved by using sigmoid function,and the partial channel connection is also adopted to reduce the computation overhead.In addition,the zero-one loss function is used to accelerate the discretization of mixed operations in the search process to solve the operation discretization problem.For the above methods,this thesis analyzed theoretically the optimization of search time,architecture performance and the effectiveness solution of DARTS defects.And the experiments are conducted on CIFAR-10 and Image Net dataset,the result shows that DNCDARTS reduces greatly the search time.It only takes 0.06 GPU days to search architecture,and achieves 2.53% test error on CIFAR-10 dataset.Compared with the state-of-art NAS algorithms,our search time is the shortest and the time overhead is reduced by 96% with high accuracy.We also carried out the experiment that verifies the transferability on Image Net with the architecture searched on CIFAR-10.