Image Super Resolution Based On Auto-encoder Networks With Joint Regularization

Super-reslution is used to estimate a latent high-resolution image based on several low-resolution images from the same scene.The development of SR has been a hot issue in computer vision field which promotes the study of SR performance.Commonly,SR uses a priori knowledge to set the constraint rule,and then obtain the optimal solution under these constraints.These prior knowledge includes smoothness,self-similarity and structural continuity.Although the existing deep learning methods successfully used in the SR problem,we find its shortcomings in three aspects:1.the existing deep networks rarely consider the use of image prior information on the model constraints,resulting in slow convergence of the model,2.the existing deep networks training has no limit on training set and neglects the impact of noise data on model,resulting in the model deviation from the optimal solution.3.The existing deep networks is generally for single-scale amplification training and short of the study of other model.This article mainly discusses these three aspects as following:1.A coupled auto-encoder super-resolution reconstruction network based on joint regularization constraints is proposed.The existing Coupled Deep Auto-Encoder(CDA)can study a mapping from LR to HR patches.However,since CDA ignores the local similarity and similarity of images,it lacks of expression ability.To avoid this,the thesis plus CDA with regularity constraints,which will enhance the performance of SR.Firstly,a SR reconstruction model based on localized all-variational regularization constraint,local full variational regularization constraint and regularization constraints similar to CDA fitting are proposed.Since the derivative of the polynomial optimization function can not be solved directly,this paper uses the Bregman method to transform the constraint function into the penalty of the function as the whole optimization.Experiments show that the adding regular constraints can improve the results and visual effect of images.2.Propose depth network training method based on data amplification and active sampling.Although regularization constraints effectively improve the performance of model reconstruction,these regularization constraints are not included in the CDA network training process.To further improve the network expression ability,this thesis first increases the amount of original training sets,adding the picture in the BSDS500 database,rotating and scaling each picture.Then,we initiatively sample the image blocks in the training library and retained the image blocks with texture features and edge details to further adjust the network parameters.Eperimental results show that the model of data augmentation training has a big improvement both in the objective evaluation index and the subjective visual effect.3.Proposed a SR reconstruction method based on multi-scale single-model.Since eisting method are single-scale model,this thesis propose a multiple-scale model by adding two,three and four times of training dataset.In the experiment,different combinations of scale training set are tested.The experimental results show that the multi-scale training set training network can improve the reconstruction effect of the network at different scales.