Research On Upsampling Method Of Depth Image

Depth data is one of the basic attributes of natural scenes.It is also very important for machines to understand natural scenes,and depth is widely used in 3D reconstruction,robotics,autonomous driving and 3DTV.However,due to the complexity of the natural scene and the limitation of depth camera manufacturing process,the resolution of depth maps captured by the depth camera are relatively low.These low-quality depth maps cannot be directly applied to related fields.Therefore,how to use image upsampling technology to recover high-quality depth maps from original low-quality depth maps has a great practical significance.The focus of depth map upsampling is to enhance the resolution while reconstructing the detailed texture structure.Since the application of depth information is often combined with color images and the depth map and color image represent the same scene,thus they have high correlation in space and texture.So the color image is introduced to the upsampling process of depth maps.However,a large number of studies have shown that this color-guided upsampling method is actually unstable—the texture copy artifacts of and the blurring edge of depth map are the two biggest problems that need to be faced.In this paper,the problems in the depth map upsampling guided by color image is analyzed,and this paper proposes a region-adaptive weight model to represent the spatial and texture structure similarities between the depth image and the color image.The model combines depth information and color information in different ways by differentiating the local region characteristics of depth maps to achieve different weight ratios.And this paper analyze the inherent causes of edge errors in the upsampling process and propose an edge correction mechanism.We combine all those with a robust global energy framework.The actual performance of the algorithm is demonstrated by multiple experiments in the simulation dataset and real dataset,At the same time,considering using color image to guide the depth image upsampling requires very fine registration,and the algorithm takes a long time.In this paper,combined with the powerful ability of convolutional neural network,a future fusion upsampling convolutional neural network combined with attention model is designed.The network does not require the guidance information of color image,and can automatically learn low-level and high-level features that cannot be designed manually.Experimental results on the simulated data set and the real data set show that the network can further improve the accuracy.