Towards Perception Assessment And Lightweight Deep Model For Super-Resolution

Single image super-resolution(SISR)aims to reconstruct a high-resolution image from its degraded counterpart,which needs to improve the clarity while increasing the number of pixels.It has a wide range of applications in high-definition television,historical data recovery,compression transmission,monitoring and security,and medical diagnosis.With the rise of convolutional neural networks and the improvement of computing resources,deep image SR has achieved unprecedented research progress in both objective indicators and visual effects,which surpasses the traditional reconstruction algorithm.Meanwhile,how to make a balance between the subjective evaluation of the image quality and the objective index,design a lightweight network with high reconstruction accuracy,and improve the generalization of the model are still research difficulties at present.This thesis studies on the problems of imbalanced perception-distortion in deep image SR networks,increased computational complexity and high memory consumption due to the expanded network capacity,as well as poor generalization of existing models on non-bicubic degraded datasets.The main innovations of this thesis are as follows:Firstly,a bi-branch image SR network based on soft-thresholding fusion is proposed.Aiming at the problem of evaluation imbalance between the SR algorithm based on objective indicators and the SR algorithms based on perception,this thesis proposes a balance scheme based on soft-thresholding fusion.To maintain a better balance between the subjective evaluation and objective indicators of the SR images,a residual memory network and a weight perception network are designed to obtain two different SR results.Then,the soft-thresholding function is adopted to fuse the two bias images,which can make the PI and RMSE compromise.The soft-thresholding fusion just needs to adjust the threshold to get intermediate balanced results rather than train the models in different regions separately,and can reduce noise.Experimental results on public SR benchmark datasets show that the proposed algorithm can obtain comparable perceptual results with other state-of-the-art SR models for 4×SR.Secondly,a lightweight SR network based on lattice block is proposed.Aiming at the problem of designing a lightweight SR network,an economic structure is proposed,which can adaptively combine residual blocks.Inspired by the lattice filter,it has a topology that can realize multiple linear combinations between two signals.A lattice block is designed which favors lightweight SR network,where two butterfly structures are applied to connect two residual blocks.Besides,it can represent multiple combination patterns of the two residual blocks by the learnable combination coefficients.The lattice block can be replaced in any SR network that uses residual blocks as the basic mapping module to achieve equivalent performance with nearly half amount of parameters and computation.Based on the lattice block,a lightweight SR network,LatticeNet,is constructed,where a backward sequential concatenation strategy is adopted to integrate contextual information from different receptive fields.Experimental results on available benchmark datasets demonstrate that the proposed method can achieve superior accuracy,while maintain relatively low computation and memory requirements.Finally,a method based on domain adaptation is proposed.Aiming at mitigating the problem ofpoor generalization ability of existing models on non-bicubic degraded datasets,domain adaptation mechanism is introduced.Although the existing deep SR models can learn powerful representations and mapping relationships from a large number of synthetic datasets,they still cannot be well adaptive to the changes in the input distribution.CORAL loss is used to align the second-order statistics of the source and target domain features.Based on LatticeNet,the domain adaptation mechanism is introduced for reducing the domain difference between the training dataset and the test dataset to compensate for the performance degradation caused by domain shift.