Point Cloud Optimization Research Based On Image Information

The popularity of low-cost RGB-D scanners is increasing on a daily basis,such as Kinect.So the depth acquisition of the objects is very convenient and quick.Nevertheless,existing scanners often cannot capture subtle details about the objects with much noises.For Kinect,the loudest noise is coming from the scanner and the external lighting environment that it will affect the next steps of 3D reconstructions.So this research is focusing on it that how to improve the depth map based on the known original depth and the RGB image from Kinect.After the depth image processing,we transform the 2-d coordinates to 3-d point cloud to testify the result of the algorithm.Firstly,we assume that the input depth and a corresponding RGB image are taken from a calibrated fixed system with the same size.Then we study the MRF(Markov Random Field)which is often used in image segmentation to achieve the depth enhancement.From the simple basis model to the later second order interpolation estimation model,we testify that the MRF can reduce the noise and maintain some features,to some extent.To improve the result of the boundary,we use the canny operator to get the boundary and add a window restrict to increase the weight of corresponding data item.The experiments show that this way can improve the quality of the point cloud.Then we turn to another way known as SFS(Shape-from-Shading).The lighting model we use can handle the natural scene illumination.It is integrated in a shape from shading like to improve the visual fidelity of the reconstructed objects,combining the depth and RGB image.Working out the parameters of this model,then we use connections between the normal vector and depth gradient to get the improved depth directly.The result illustrates that this way can support the qualitative and quantitative improvement of the depth.The two ways are familiar on the model building and solution procedure.we will give all the detailed process in the following chapters.