A Planar Object Tracking Algorithm Based On Multiple 2D Constraints And 3D Optimization

Planar object tracking is an important 3D tracking technology in computer vision.Its core technology is how to track objects with planar structures in various complex environments effectively and accurately.It has been widely used in many fields such as augmented reality,unmanned driving and robotics,etc.At present,the planar object tracking algorithm has achieved good results,but in face of harsh tracking conditions,the existing algorithms still cannot be adapt to various challenging interference factors perfectly.For some scenes with complex background texture,drastic changes in viewpoint,missing information of object and large appearance deformation,the stability of the algorithm is insufficient,and the accuracy cannot meet actual needs.A planar object tracking algorithm based on multiple 2D constraints and 3D optimization is proposed in this thesis.This algorithm mainly includes steps such as optical flow initial tracking,line segments extraction,multiple constraints construction,homography matrix correction and 3D optimization.Multiple constraints and 3D optimization are the focus of this thesis among them.The details are as follows:(1)In order to track the edge line segments of the planar object,the geometric attributes of the line segment,the coincidence degree between the line segments and the line segment feature are used to design a multi-constraint screening framework.First,the discriminant formula of geometric attributes is established in this thesis,so the interference line segments are excluded from the edge candidate set according to the combination probability.Next,the principle of definite integral is used to define the coincidence degree function,and the candidate line segments are further filtered by the edge coincidence degree of adjacent frames to avoid a few broken line segments from being mismatched due to similar background colors.Then,the color feature of the Lab model and the pixel gradient value are used to construct a line segment descriptor with high recognition and uniqueness,and the scale factor and weight parameter are added to make it better adapt to the size changes of the object appearance.Finally,edge matching is achieved by combining the constraints above,and edge repair and shape correction are achieved by homography matrix.(2)In order to reduce the negative influence of blurred edges of image and missing object information on the 2D constraints in the actual tracking process,the suppose that the position of the planar object in the real world is fixed is assumed.An optimization strategy of using spatial 3D information to correct the 2D position of the object is proposed based on the property.First,two vanishing points are obtained by the fitting lines.Then,the single-frame camera parameters are solved based on the vanishing points,and the 3D coordinate of the planar object is reconstructed according to the camera parameters.Finally,the 3D points in the space are reprojected to obtain the final tracking results.(3)The latest public POT data set is used to evaluate the validity,accuracy and robustness of the algorithm in this thesis.The proposed algorithm is compared with the current mainstream tracking algorithms Gracker,GO-ESM,etc.in various complex scenes such as scale change,rotation,perspective distortion,occlusion and out-of-view.Then the qualitative and quantitative analysis are made from the visual effects and error data.The experimental results show that,for all kinds of indoor and outdoor scenes,the algorithm can achieve valid,accurate and stable tracking.The overall accuracy is improved by 39%compared to Gracker.It also has certain advantages compared with other methods.