Research On Image Super-resolution Based On Neural Architecture Search

Super-resolution(SR)reconstruction,including scale reconstruction and motion deblurring reconstruction,refers to the process of recovering shape and clear images from low-resolution or blurred images,which can be applied to a number of domains such as high definition videos,medical imaging,security,computational photography.SR can also be responsible for pre-processing of high-dimensional computer vision applications,such as deblurring in object de-tection and segmentation,text enhancement and reticle removal in character recognition,etc.Most of the deep learning-based image super-resolution reconstruction methods are de-signed manually.The problem with manually adjusted networks based on experience and mul-tiple experiments is that different networks need to be trained in different super-resolution sce-narios,and network structure,loss function and other aspect differ a lot.In order to make up for the deficiency of the above manual designed network,the idea of using Neural Architecture Search(NAS)to sample image super-resolution reconstruction network structure has emerged in recent years.However,loss function,color space,training methods and other inventions have also introduced artificial prior knowledge in networks searched by NAS.In addition,genetic algorithm is used in most search methods,which leads to a large search space and long search time.To deal with this problem,a Neural Component Search(NCS)method is proposed in this thesis,which expands the representation of search space and accelerates the search speed by using the parameter sharing method.The main research contents of this thesis are as follows:(1)The Neural Component Search method was designed.To deal with the disadvantage of NAS that the search space is only limited to network structure,the composition of loss functions and color space are also taken sa part of search space in the NCS method.The components of loss functions include pixel loss,individual noise loss and adversarial loss.In addition,the choice of adversarial loss also determines the training methods of the searched SR network.(2)The proposed Neural Component Search method was used to search the image super-resolution reconstruction network under specific constrains to solve the cross-scale super-resolution problem,and the search space of the shared generator and shared discriminator was designed.The shared generator is composed of four modules: Shallow Feature Extraction,Adaptive Feature Extraction,Deep Feature Extraction and Upsample.The shared discriminator is designed as a relatively compact model with low degree of freedom,which is composed of five discriminator modules and full connection layer.At the same time,search space of four loss functions and two color space is designed under the above settings.Different networks are searched according to different computational constrains,and the search efficiency and perfor-mance of SR models searched by the proposed NCS method were verified by experiments.(3)An end-to-end motion deblurring network was designed to solve the same-scale super-resolution problem.Besides,a motion blurred image data simulation method was designed to expand the training set.The nature of the simulation method is to simulate the random trajec-tory of the photographed scene relative to the center of the imaging sensor.The motion blur kernel is generated through the motion trajectory to obtain the blurred image.The inference speed and performance of the proposed end-to-end motion deblurring network were verified by experiments.(4)The combination the super-resolution technology and human action recognition was researched to improve the accuracy of action classification.The detection accuracy of human skeleton was improved by super-resolution frame processing on the detection video,so as to im-prove the construction accuracy of time-space skeleton graph and realize the improvement of accuracy of action classification.Experiments showed that the super=resolution frame process-ing could improve the accuracy of human action recognition models in both Kinetics validation set and fall-down detection applications in real natural scene.