Research On The Key Technology About Structure Reconstruction And Localization Based On Monocular Vision

In this paper, several key technologies that related to 3D structure reconstruction and camera pose solution are discussed. These technologies include local invariant feature detection, description and matching, fast and robust essential matrix estimation, and PNP(Perspective-N-Points) based camera absolute pose solution.In the section on local invariant feature detection, description and matching, the state-of-the-art methods are first reviewed, including Harris corner, SUSAN(Smallest Univalue Segment Assimilating Nucleus) corner, SIFT(Scale Invariant Feature Transform) and SURF(Speed Up Robust Feature). To improve computational efficiency, several binary features have been proposed in recent years, including FAST(Feature from Accelerated Segment Test), AGAST(Adaptive and Generic corner detection based on the Accelerated Segment Test), BRIEF(Binary Robust Independent Elementary Features), ORB(Oriented FAST and Rotated BRIEF) and BRISK(Binary Robust Invariant Scalable Keypoints). In this paper, a new feature(namely Speed-up BRISK, SBRISK) is proposed to integrate the advantages of the existing binary features. In the keypoint detection stage, cascaded AGAST is used to detect keypoints with scale space searching. In the keypoint description stage, a circular symmetric constellation is used to describe the pattern of a selected keypoint. To adapt to the characteristic orientation of keypoint, SBRISK shifts the binary vector rather than rotating the image pattern or constellation(which has been used by many existing descriptors). Different from BRISK, SBRISK classifies keypoints into bright patterns and dark patterns. Comparison is performed only within the same pattern. Meanwhile, a special refinement scheme is imposed upon the putative matching results to improve the matching accuracy. Experimental results show that SBRISK outperforms BRISK with less memory consumption and about 30%~40% saving in computational cost.In the essential matrix estimation stage, a pinhole perspective projection model is introduced to induce the fundamental matrix and essential matrix. The property and solution to these matrices are also discussed. To accelerate the RANSAC process for essential matrix estimation with severe outliers, two special modifications about RANSAC are proposed.In the verification stage, not the correspondences are used to verify the solved matrix but the singular values of the solved matrix are directly used to test its reliability.Once a plausible estimation is obtained, the obvious outliers are eliminated from the correspondences set. This process enhances the percentages of inliers in the remaining correspondence set, which therefore accelerates the RANSAC procedure. This method is called outliers elimination based RANSAC(OE-RANSAC). Experimental results on both synthetic and real data shown that OE-RANSAC is ten times faster than the original RANSAC.To remove the affect of critical configurations coming from special motion and scene patterns in the algebraic solution of essential matrix, a new top-down and robust method is proposed based on PSO(Particle Swarm Optimization). Different from the traditional data-driven methods, the PSO based method can be considered as model-driven. Given five or more corresponding points, a stable solution can always be calculated with two calibrated views by the proposed method, even with a large proportion of outliers. In PSO based method, an essential matrix is first built, the fitness between this matrix and the correspondences set is then evaluated. PSO is introduced to find out the particle with the highest fitness. Several modifications are made to keep the original PSO away from being early trapped to local minima and to accelerate the convergence rate. They are crossover operator, randomly weighted global best position and randomized search velocity. Experimental results show that the proposed method achieves a better or comparable precision compared to the classical algebraic methods under almost all circumstances.Finally, the PNP based camera pose solution methods with N(28)3,N(28)4,N(29)4 are reviewed in this paper. After the review of the state-of-the-art PNP methods, an extendable RA(Rotation Averaging) based PNP method is proposed. It directly averages the poses solved by P3 P and suppresses the noise in each pose. In this section, three P3 P methods are compared. It is shown that their results are comparable. Consequently, the computational efficiency becomes the most important parameter for a P3 P implement. It is shown that Kneip’s method outperforms the others in terms of computational efficiency. For P4 P implementation, RA based method achieves the best performance. Comparative results of the four PNP methods show that RA achieves comparable performance to the EPNP method proposed by Lepetit in general scenes. However, the RA method is free from planar degeneration and obtains a stable solution under any circumstance.