Research On Robust 3D Object Tracking For Augmented Reality

Estimating the pose of a rigid object,including its position and orientation,is a fundamental problem in computer vision,and plays an importent role in augmented reality.In augmented reality(AR)environment,in order to ensure the geometry consistency between the rendered virtual object and the target real object,AR system needs to track the pose of the target object in real-time.3D object tracking is the key technology to continuously estimate the pose of the target object in video frames.Computer vision-based 3D object tracking has developed rapidly because it is a non-invasive,accurate and low-cost solution.With the wide spread of portable devices and the development of computing capability,3D object tracking based on monocular color cameras becomes the research hotspot,and is widely applied in gaming entertainment,industrial manufactureing,education,healthcare,etc.Monocular 3D object tracking aims to estimate the pose of the object with respect to the camera,based on the consecutive video frames.Although many different approaches have been proposed and the state-of-the-art is rapidly evolv-ing in recent years,implementing robust and accurate 3D object tracking remains a challenging task,and there are many problems to be solved.These problems are caused by physical feature of the object,such as textureless surface and symmetrical structure,and by features of the surrounding environment,such as heavily cluttered backgrounds,partial occlusions,ambiguous colors and dynamic lighting.Above conditions may easily cause the problems of tracking drift and tracking failure.To tackle these problems,this thesis conducts an in-depth study on monocular 3D object tracking using color video frames.Firstly,we study adaptive fusion of region color feature and object contour feature in the process of 3D object tracking.Secondly,we study to improve robustness of edge-based 3D object tracking method against cluttered backgrounds and partial occlusions with edge confidence.Thirdly,we study to improve robustness of region-based tracking method against partial occlusions and ambiguous colors with contour constraints.The main contributions of this thesis are as follows:1.Proposing a 3D object tracking method based on adaptive feature fusion.Recent region-based 3D object tracking methods only rely on the region color feature around the object contour,but ignore the object contour feature.How-ever,edge-based 3D object tracking methods only rely on the object contour feature,but ignore the region color feature.We propose to adaptively fuse the region color feature and the object contour feature using search lines around the object contour,so as to make them complement each other and futher improve the tracking performance in more complex environment.Firstly,the region color feature and the object contour feature are extracted from the search lines sam-pled around the projected model contour.Then the energy function is defined based on adaptively weighted region color feature and object contour feature,and the differentials of this energy function with respect to pose parameters are derived.Finally,the optimal pose is obtained via Levenberg-Marquardt solver.To better deal with fast motion of object and camera,a coarse-tofine iteratively pose optimization strategy is applied on multi-scale video frames.Qualitative and quantitative experiments demonstrate that the proposed method improves robustness and accuracy with respect to cluttered backgrounds and motion blur effect.2.Proposing a 3D object tracking method based on edge confidence.Previous edge-based 3D object tracking methods are prone to fail in cases of cluttered backgrounds and partial occlusions,because many false object contour points caused by complex environment can easily deviate pose optimizaton.We pro-pose a novel edge-based method to tackle this problem.To search the object contour points,foreground and background clutter points are first filtered out using region color cue,then object contour points are accurately searched by maximizing their edge confidence which combines region color and distance cues.Furthermore,the edge confidence is integrated into the edge-based energy func-tion to reduce the influence of false contour points.We also extend our method t,o multi-object tracking which can handle mutual occlusions.We evaluate the tracking performance on two challenging public datasets.The experimental re-sults show that our method improves robustness and accuracy against cluttered backgrounds and partial occlusions.3.Proposing a 3D object tracking method based on pixel-wise weighted op-timization.Previous region-based 3D object tracking methods heavily rely on region color statistics,but partial occlusions and ambiguous colors may ruin the region color statistics and lead to tracking drift and tracking failure.We propose a novel region-based method to tackle this problem.Firstly,we define a novel region-based cost function using search lines around the object contour,which is more efficient than the previous region-based cost functions using signed distance transform,and in the meantime can deal with partial occlusions and ambiguous colors more effectively.Secondly,we propose a pixel-wise weight function based on color and distance constraints of the object contour points,and integrate it into the proposed region-based cost function to reduce the negative impact of par-tial occlusions and ambiguous colors.We evaluate the tracking performance on two challenging public datasets.The experimental results show that our method improves robustness and accuracy against partial occlusions and ambiguous col-ors.