Study On Depth Sensing With High Accuracy And Robustness

Depth sensing techniques play an important role in enormous fields such as virtual reality,augmented reality,automatic drive,artificial intelligence,etc.Structured-light(SL)depth sensing techniques have been widely studied by academics and industry researchers,due to the advantages in terms of high accuracy,large resolution and low cost.In industrial applications,the performance of depth sensing will be affected by various factors,such as disturbances of ambient light,shadings or different kinds of colors,highly contrasted albedos,rich depth variations,abrupt depth changes or tiny constructions in a scene.It is difficult to obtain satisfactory results of depth sensing under these disturbances,because the codes constructed by colors,intensities,shapes or frequencies in a projected pattern are easily disturbed.Robust and accurate depth sensing is one of the most significant and challenging tasks in SL field.In this paper,we propose three novel methods to address this challenging task from three aspects: pattern design,feature detection,and correspondence retrieval.First,we propose a SL depth sensing method with a single-shot coding-free binary grid pattern.Binary lines are robust to the disturbances of ambient light,uneven albedos or colors in scenes.In addition,binary grid pattern is simply constructed,thus it can be easily projected by low-cost instruments and is suitable for many real-world applications.The main challenge for depth sensing with the coding-free pattern is how to retrieve correspondence between the projected and captured pattern.A graph based topological labelling algorithm is proposed to determine the topological coordinates of the intersections in the grid.Then the correspondence is retrieved by exploiting the topology of grid and the epipolar constraint.We also propose a coarse-to-fine line detection method,which is capable of suppressing ambient light noise and locating the lines in captured grids with high precision.The proposed technique is a general solution of correspondence retrieval,which can also be used for both binary and chromatic grid patterns.Experimental results show that the proposed technique performed better than the popular RGB-D cameras Kinect and SwissRanger in terms of precision.Compared with the traditional single-shot techniques with a complicated pattern,the proposed technique significantly improved the robustness and achieved comparable precision.Second,to further improve the robustness of depth sensing to edges and scenes with abrupt depth changes,we designed a monochromatic maze-like pattern and proposed a maximum a posteriori estimation-based correspondence retrieval method.The maze-like pattern is more robust to ambient light,colors,edges and abrupt depth changes in scenes than traditional patterns.The proposed maximum a posteriori estimation-based correspondence retrieval method uses the inherent spatial correlations among lines during correspondence retrieval.The correspondence of an incomplete line is determined by both the Euclidean distance of its code and the prior results of the correspondences of its neighboring well-detected lines.Experimental results demonstrated that the proposed system performed better than the popular RGB-D cameras and traditional single-shot techniques in terms of accuracy and robustness,especially on challenging scenes.Third,we reinvestigated one of the most fundamental problems in SL depth sensing field: correspondence retrieval of features between a projected pattern and captured image.We generalized all kinds of features in a captured image into three components: position,code,and decoding confidence.Then the global optimum correspondence retrieval of the features was formulated by maximizing a conditional probability of correspondences given observed features,which was depicted by a Bayesian network.Different from traditional “code only-matching” based correspondence retrieval methods,the proposed Bayesian network based method uses the positional correlation of correspondences of neighboring features,namely,the correspondences of poorly detected features are estimated with the aid of the correspondences of well detected features.The method performed especially well on challenging scenes with rich depth variations,abrupt depth changes,edges,etc.Experiments showed the proposed method significantly improved the correspondence accuracy on challenging scenes,compared with traditional code only-matching based correspondence retrieval methods.