Research And Application On Neural Network Architecture Search Based On Evolutionary Algorithm

In recent years,deep learning has made great development.Convolutional Neural Network(CNN)is widely used in various fields of computer vision,such as image classification,object detection,image segmentation,and image restoration,making outstanding contributions.Most of CNN is based on manually-designed neural network architectures,which require specialized domain knowledge and repeated experimental iterations,greatly increasing the difficulty of designing excellent network architecture.To address this problem,Neural Architecture Search(NAS)has gradually attracted the attention.NAS algorithms can automatically search for network architectures,reducing human involvement in the design process,thus helping to further promote the application of neural networks.However,NAS algorithms generally cost long search times and high computational resource requirements.Moreover,most NAS algorithms currently focus on image classification,there are improvements for the NAS algorithms in image denoising.In addition,multi-objective NAS algorithms are more useful than single-objective ones for the application in real scenarios.Therefore,new NAS algorithms based on evolution algorithm are proposed in this paper to address the problems mentioned above.(1)A low-cost NAS(LoNAS)algorithm is proposed to address the problems of long search times and high computational resource requirements in existing NAS algorithms for image classification,as well as the problems of single-objective NAS algorithms that are not conducive to real-world applications.First,a variable-length encoding strategy based on a new network block(Reg Block)is proposed to construct multi-objective network architectures with high accuracy and low parameter.Then,a non-training proxy strategy based on Neural Tangent Kernel(NTK)is proposed to accelerate the search for NAS,effectively reducing search time and computational resource requirements.Finally,a three-stage evolution algorithm based on a multiple-criterion environmental selection strategy and network block mutation are designed to balance the exploitation and exploration of the search algorithm and help find better solutions.Experimental results show that by considering the classification accuracy and parameter comprehensively,compared with other SOTA algorithms,competitive network architectures can be found by LoNAS.Moreover,LoNAS can significantly reduce search time and computational resource costs.In addition,the network architectures searched by LoNAS have good generalization performance on multiple datasets.LoNAS demonstrated reliable accuracy in gender classification on the IMDB-WIKI face dataset,while requiring fewer parameters,which indicates that LoNAS has good application performance.(2)Regarding the field of image denoising,a high-performance denoising NAS algorithm(DeNAS)is proposed to improve the denoising performance of neural networks.Firstly,by designing a variable-length symmetric encoding strategy based on U-Net and attention mechanisms,which can construct a search space containing high-performance denoising network architectures.Second,a regression mapping function based on short-term training is proposed.By introducing the parameter of the network architecture,the denoising performance of the network under full-term training and short-term epoch as prior knowledge,the search time and search resource requirements can be effectively reduced.At the same time,the reliability of short-term training strategies can be improved.Finally,a genetic evolution algorithm based on dynamic allocation of parameter weights is designed to effectively improve the denoising performance of the population.Experimental results show that DeNAS can effectively accelerate the search for NAS.The denosing performance of network architectures found by DeNAS can surpass other SOTA algorithms on multiple datasets with different noise level.Finally,DeNAS algorithm can discover network architecture with good denoising performance on the SIDD dataset,showing reliable denoising ability on real image noise.