Researches On Image And Video Restoration Based On Regularization Optimization

Recent years have witnessed the explosive development of the signal acquisition technology and the computer capability,which makes the multimedia applications car-ried by image and video contents getting vast attention and being rapidly popularized.Image/video based applications begin to play indispensable roles in nowadays consumer electronic area(such as digital camera,smart phone,and so on)and professional pho-tography area(such as medical imaging,security surveillance,and so on).However,quality degradation usually occurs in the acquired image/video signal,during the cap-turing procedure.This phenomenon is due to certain subjective or objective reasons like circumstance noise,device defect,the ability of photographer,data compression,and so on.The quality degradation of image/video will not only influence the intuitive experience of observer,but also damage the effectiveness and accuracy of subsequent processing tasks(like segmentation,recognition,and understanding).Therefore,it is obvious that how to recover a result close to the ground truth visual contents from a de-graded image/video is one of the most fundamental and important issue in multimedia processing community.Most image/video degradation phenomena can be modeled as a function mapping of the ground truth signal plus random noise.There are usually numerous recovered results that can fit the degradation model,due to the ambiguity of the function mapping in the degradation model and the extra disturbance of the random noise.Though these recovered results fit the mathematical model well,most of them are ill-posed solutions that are quite different from the ground truth image/video.In order to resolve this prob-lem,prior information that reflects the intrinsic properties of image/video has to be in-corporated in the restoration process.Thus,we can regularize the restoration procedure to find the final result within a proper region near to the ground truth signal.The main idea to achieve this goal in academy and industry is to formulate the restoration task as an optimization problem,where certain regularization term is introduced to regularize the final solution.By this way,certain appropriate properties can be guaranteed in the optimal solution when we solving the corresponding problem,which finally leads to a recovered result close to the ground truth enough.In the past twenty years,extensive theoretical analysis and practical applications on formulating image/video regulariza-tion model have been proposed by researchers.However,image/video usually exhibit diverse prior characteristics in multi-element aspect,multi-dimension aspect,different contents,and different regularizing mechanism,which have not been well investigated.This dissertation mainly focuses on the regularization modeling for image/video restoration and discusses how to construct and solve regularized optimization problems in different application scenarios based on the prior properties that image/video exhibit in multi-element aspect,multi-dimension aspect,different contents,and different regu-larizing mechanism.This dissertation conducts investigations on four aspects:1)group sparse representation of similar image patches,2)spatio-temporal correlation model-ing for different video component,3)the non-stationary and inhomogenous statistical characteristics modeling of different image contents,and 4)generic modeling based on regularizing mechanism analysis,all of which aim at improving the result's quality of image/video restoration.The main innovations and contributions of this dissertation are listed as follows.1.This dissertation proposes a novel restoration scheme for the JPEG deblocking problem,based on group sparse representation model of similar image patches.On one hand,this dissertation develops the original sparse representation model into the patchwise group-sparse representation model and employs it as regular-ization term to resolve the discontinuous blocking effects in JPEG-compressed images.This modeling is inspired by the non-local similarity that an image ex-hibits in its patch scale.By combining it with an indicator function,the degra-dation mechanism of JPEG compression and the essential prior information on image patch scale can both be formulated accurately,which benefits the quality-improvement of the reconstructed image compared with other state-of-the-art methods.2.This dissertation develops a two-stage decomposition scheme for detecting the defects in the old-films(such as blotches and scratches).Firstly,based on the distinctive correlation properties of different video components in the spatial di-mension,a cartoon + texture decomposition model is constructed to separate the blotches and scratches into different components.Secondly,each component is further decomposed using a low-rank + sparse regularization model,which is based on the correlation differences between video contents and defects in tem-poral dimension.With this decomposition,the blotches and scratches can be sepa-rated from the original video contents and then be used to labeled old-film defects efficiently.The testing results demonstrate that the proposed scheme outperforms other methods in detecting defects of complex shapes and varying sizes.3.According to the non-stationary and inhomogenous statistical characteristics that different contents of an image exhibit,an content-adaptive modification of total generalized variation model is proposed in this dissertation.Firstly,this disser-tation analyzes the inherent relationship between regularized optimization prob-lem and maximum Baysian posterior probability scheme,based on the Baysian probability theory.Secondly,the image prior probability model assumption be-hind total generalized variation regularization is deduced following this idea.Be-sides,the quantitative relationship between the weighting parameters of the total generalized variation and the parameters of its corresponding prior model is fur-ther derived.Finally,with the robust and effective statistics-estimating strategy we develop,the parameters of the prior model can be estimated,as well as the content-based regularizing parameters.With these modifications,this new model can tune its weighting parameters adaptively based on the image contents and thus generates restoration results with better quality compared with classical variation based models.4.Based on the measuring®ularizing mechanism,this dissertation analyzes both the regularizing mechanism and prior assumption behind the original non-local total variation,and hence proposes a high-order non-local total variation model for video restoration.With experimental observation and theoretical anal-ysis,it is found that the prior assumption inside non-local total variation model cannot model the complicated value-changing properties of diverse video con-tents accurately.To resolve this problem,mean-value approximations for non-local gradients are introduced as auxiliary parameters in this dissertation.More-over,second-order non-local regularization is further constructed on these mean-value approximations.Thus,a more flexible and more accurate regularization model is established for video restoration.With the adaptive parameter tuning strategy we introduce,this new model can further adjust its regularizing strength according to the latent visual contents.Extensive experiments on multiple video restoration problems demonstrate that the new model proposed in this dissertation outperforms other state-of-the-art methods.