Intrinsic Decomposition And Depth Refinement For Single RGB-D Image

Intrinsic image decomposition aims at decomposing an image into several compo-nent images which separately describe intrinsic properties of objects and light effects depicted in the scene,which reflect the real physical world in the image.The most common method is to decompose an image into a reflectance image and a shading im-age.Each pixel in reflectance image represents the reflectance property of objects and shading image encodes every position's light information.Many computer vision algorithms such as segmentation,recognition and motion estimation are greatly influ-enced by light effects in the image.Performance of these algorithms will improve a lot if the material characteristics under invariant light can be reliably estimated.However,high ill-posedness of this task limits the development of related algorithms.In recent years,the commercialization of depth camera such as Kinect,inspired people to utilize depth information with RGB data together to improve computer vision algorithms in-cluding pose estimation,scene parsing^[1]and so on.TOF camera and Microsoft Kinect adopt infrared ranging to obtain depth value,and they often produce holes in the edge and occlusion areas,which damages the quality of depth image a lot^[2].Therefore depth map refinement is a hot topic and has been widely studied.Although there are many algorithms being able to improve the quality of depth map for consumer depth cam-eras,there are still some problems,such as texture interference,which greatly limits the performance of depth refinement algorithms.In this paper,we propose a new intrinsic decomposition model by which reflectance and shading components can be recovered.Given a single RGB image and its initial depth map,we fist decompose it into four subcomponents:reflectance,direct irradi-ance,other irradiance and illumination color,then we recover shading using the latter three.A sparsity prior and a non-local prior are jointly imposed on reflectance com-ponent.To be specific,the latter is weighted by a bilateral kernel which enables the proposed model to exploit structural correlation from a larger neighborhood centered at each pixel.Based on our observation that shading images mainly consist of smooth regions represented by curves and their gradient fields are sparse,we utilize L1-norm to model the direct irradiance component that is a main constituent of shading.The alternating direction method under the augmented lagrangian multiplier?ADM-ALM?framework is adopted to solve the model.The performance of the proposed method is demonstrated by experiment results on both synthetic and real-world dataset when com-pared with state-of-the-art methods.Besides,the proposed method is robust to noise.With recovered intrinsic components,we can jointly utilize reflectance and shading to refine initial depth map by considering various characteristics of different parts in the scene.It shows that we can recover more geometry details and effectively eliminate texture artifacts.The main contributions of this work are summarized as:1.Sparse constraints are imposed on the reflectance component and the direct irra-diance sub-component derived from shading component.Based on our observation and analysis,the local finite differences of reflectance and shading images present Laplacian distributions,and can be well-modeled by using ₁-norm as a regularizer.2.A non-local prior that considers non-local similarity weighted by a bilateral kernel is designed to fully exploit structural correlation in the reflectance component,remedying the short-sighted local correlation in former methods.3.An efficient depth refinement framework based the proposed intrinsic decompo-sition method,which can recover more geometry details and avoid introducing texture-copy problems to the greatest extent.