| In recent years,deep neural networks have been successful in various fields such as image recognition,speech recognition and machine translation.However,the current network architecture is designed by human experts manually,which is time-consuming and computation-intensive,and requires sufficient expertise.Thus,neural network architecture search method is getting more and more attention.However,the cost of evaluating the performance of each network is considerable.Therefore,selecting the search space,finding the appropriate search method and evaluation method are big problems that network architecture search needs to face.The proposal of differentiable neural architecture search greatly reduces the resource consumption of search.The gradient method is used to optimize the weight of network architecture,and the optimal network architecture is searched by selecting the candidate operations with higher weight of architecture,so that the network model can be searched quickly.However,due to the unfair competition between operations in the search space,the network with abnormal structure will be found.In addition,because of the relatively immobilized search space,the obtained network model lacks diversity.Aiming at the above problems,this thesis carries out the following research:(1)Aiming at the problem of network performance crash caused by too many skip connections in the search process of differentiable neural network architecture,a differentiable neural architecture search method based on attention mechanism is researched and designed.By using the attention mechanism,the attention weights of channels are obtained and the characteristic information is strengthened.In addition,the partial channel connection is proposed.The channels with greater attention weight are sent into the operation space for calculation,while the other channels are directly contacted with the output,which reduces the memory occupancy and improves the search efficiency and memory utilization.In addition,the algorithm weakens the unfair competition between operations to search the network structure stably.The experimental results show that the classification error rate of the proposed algorithm on CIFAR-10 and CIFAR-100 is 2.46% and 17.06%,which can quickly and stably find the network architecture with better performance.(2)Aiming at the problem of architecture anomaly and lack of diversity in differentiable neural architecture search,an evolutionary neural architecture search method based on gradient guidance is researched and designed.By designing a supernet,the accuracy and diversity of the final model can be guaranteed while the search space can be reduced.The global and local search capabilities of the algorithm are enhanced by evolutionary computing and gradient search.Parameter sharing strategies are used to evaluate different architectures during training.Subnets can directly inherit the weight of hypernets,so that fitness values can be obtained directly without training them,which greatly improves the search efficiency.The experimental results show that the test errors of the searched architectures on the CIFAR-10/CIFAR-100/Image Net are 2.47%,17.17% and 23.9%,respectively.This algorithm can effectively accelerate the training and evaluation process and obtain high performance network architecture. |
