Research On Automatic Search Algorithm Of Convolutional Neural Network Architecture

In recent years,with the extensive research and application of artificial intelligence technology,deep learning has been well used by virtue of the advantages of extracting and learning features.Convolutional neural network(CNN)plays an important role in deep learning.However,it is not easy to manually design the CNN architecture,which requires a lot of CNN domain expertise and work experience,this limits the application of CNN.In order to automatically design CNN architectures with superior performance to solve the problem of manually designing CNN architectures,Neural Architecture Search(NAS)algorithms are proposed.NAS algorithms reduce the cycle and cost of manually designing CNN.However,NAS algorithms often require complete training and verification of each CNN architecture,and most NAS algorithms only consider the single objective of the verification accuracy of the CNN architecture,which costs a lot of computational resources and time for the computational search process.By studying the CNN architecture and NAS algorithms,this paper proposes improved algorithms to optimize the performance of the NAS algorithms.The following work is carried out to address the above problems of the NAS algorithms:(1)Aiming at the problem that the search process of existing NAS algorithms takes a long time and occupies many computing resources,this paper proposes an efficient and flexible NAS algorithm.Based on the variable-length coding method,this paper uses the depth modules Res Net Block and Dense Net Block,the width module Inception Block and the lightweight module Mobile Block to construct a flexible search space,so that the algorithm can automatically design a flexible network architecture according to different application scenarios.In this paper,the search time is greatly shortened by training the validation model under the divided dataset.In order to make up for the unreliability caused by dividing the dataset,this paper uses a multi-objective fitness correction method,which comprehensively considers the three objectives of model verification accuracy,parameters and computational complexity through the multi-objective method.The experimental results on the datasets CIFAR-10 and CIFAR-100 indicate that the algorithm proposed in this paper can design a superior and reliable CNN in a relatively short time.(2)Aiming at the problem of depth gap in the gradient-based differentiable NAS algorithm proposed in the past three years,this paper proposes a fast and progressive NAS algorithm based on the differentiable NAS algorithm.On this basis,the edge selection factor and greedy index are used as edge selection criteria to improve the correlation between the search stage and the evaluation stage.In order to shorten the NAS search time,this paper proposes a progressive dataset division algorithm,which divides the dataset in stages to shorten the search time.The experimental results of the fast and progressive algorithm on two CIFAR datasets indicate that the CNN architecture searched by the proposed algorithm has high search relevance.The stability is improved,and the search time is greatly reduced.(3)The existing NAS algorithms often take a long time due to the training and verification of the CNN architecture.In addition,most NAS algorithms only consider the accuracy of model verification,ignoring the problem of parameters,and the searched CNN architecture is relatively simple.To solve these problems,this paper proposes a multi-objective evolutionary NAS algorithm based on differentiable search space.This algorithm avoids the training and verification of the CNN architecture from scratch through the training-free NTK evaluation method,which greatly shortens the search time.In addition,this paper obtains the Pareto front architectures through multi-objective genetic iteration by synthesizing the two objectives of the verification accuracy and parameters of the CNN architecture.The experimental results on the dataset CIFAR-10 indicate that the algorithm proposed in this paper can search for a series of CNN architectures with superior performance in a very short time.To sum up,this paper proposes three fast automatic NAS algorithms for the problems of NAS algorithms that take a long time and occupy a lot of computing resources.So that the NAS algorithms can be better applied to practical tasks.