Improving The Efficiency And Robustness Of Guided Image Filtering And The Applications

Image smoothing has been a fundamental process for a variety of ap-plications in both computer vision and computational graphics,e.g.,image detail enhancement,high dynamic range?HDR?tone mapping,clip-art image compression artifacts removal,image colorization,image texture removal,guided depth map restoration,etc.Although numerous of approaches have been proposed in recent decades,challenging issues also remain in each task that need to be solved:gradient reversals and halos in image detail enhancement and HDR tone mapping,texture copies and blurring depth discontinuities in guided depth map restoration,etc.Generally,for some tasks,there is tradeoff between smoothing quality and processing speed.For example,in image detail enhancement and HDR tone mapping,weighted average based methods can be very fast,but they are prone to produce results containing gradient reversals and halos.These artifacts are better handled by global optimization based methods,however,this is achieved at the ex-pense of much higher computational cost.How to handle these artifacts but remain processing efficiency is a challenging issue.In guided depth map restoration,due to the structure inconsistency between the guidance image and depth map,the restored depth maps can suffer from texture copies and blurring depth discontinuities.How to handle the structure inconsistency remains a challenging issue in guided depth map restoration.In this thesis,we explicitly address these challenges for fast and robust image smoothing.Specifically,this thesis makes two contributions for fast image smoothing and two contributions for robust image smoothing,detailed as follows:First,we propose a new smoothing operator named semi-global weighted least squares?SG-WLS?.It can well handel gradient reversals and halos in both image detail enhancement and HDR tone mapping.It is also able to achieve close performance to the original WLS model^[1]in several challenging tasks.Yet,our SG-WLS is?20 times faster than the original WLS model.For an image of M×N,the memory cost of SG-WLS is at most at the magnitude of max{1/M,1/N}of that of the original WLS model.Our approach is able to achieve high smoothing quality while maintain the processing efficiency.Second,we propose a new global optimization based method,which is called iterative least squares?ILS?,for efficient edge-preserving image smoothing.It can achieve state-of-the-art performance and produce results free of gradient reversals and halos.Meanwhile,it can run at a much lower computational cost and is able to run faster than the state-of-the-art weighted average based one.The ILS is able to process 1024×2048 RGB color images at the rate of 23fps with the help of GPU acceleration.The ILS is also flexible.With slight modifications,it is capable of more applications that require different smoothing properties.Third,we propose a new general variable bandwidth weighting scheme to both suppress texture copy artifacts and preserve depth discontinuities,which is applicable to a number of existing approaches.The proposed method can not only boost the performance of the existing methods but also speed up the upsampling process for about 2×?5×.Experimental results show the effectiveness and efficiency of the proposed method in suppressing texture copy artifacts,preserving depth discontinuities and reducing the computational cost at the same time.Finally,we propose a robust optimization framework to handel the structure inconsistency issue in guided depth map restoration.By analyz-ing the proposed model through fixed equation technique,we prove that the proposed model can make use of the property of both the guidance image and the updated depth map in each iteration,which helps to properly handle the inconsistency issue.The proposed method performs well in suppressing texture copy artifacts.Moreover,it can better preserve sharp depth discon-tinuities than previous heuristic weighting schemes especially for real data.We validate the effectiveness of the proposed method through comprehensive experimental results of both simulated data and real data.