Real-Time Visual Localization With A Single Camera

While vision-based self-localization has been the core method for autonomous navigation for mobile robots, this approach suffers from three major difficulties, that are, how to design a robust vision system that could be applied in dynamic natural environment, how to restore the depth information with a single camera in order to estimate the 3D pose of the robot, and how to achieve real-time performance in order to catch up with the high speed and smartness of the moving robots. In this paper, we study deep into this research topic, and present a visual recognition system that could real-timely calculate the relative 3D pose of a hand-held camera with respect to coplanar visual landmarks.Firstly, we review the state of arts of current vision localization and navigation algorithms, then present the framework of our real-time visual localization algorithm, which combines object recognition with local invariant features, feature tracking and pose estimation. Firstly it detects features from live video stream, finds objects previously learned off-line, then real-timely tracks the recognized targets across video frames, and at the same time, calculates the 3D pose of the camera with respect to landmarks in the scene. In addition, we have fully considered the intrinsic connections between the three modules, and introduce the concept of parallel computing to maximize the run-time performance.Next, we propose our Harris-SIFT feature detector, including principles, merits and improvement compared with SIFT. After that, we details the Harris-SIFT based recognition system, which is composed of a visual landmark database, feature detector, approximate nearest neighbor searching, consistency checking, and evaluation of recognition results. Experiment shows that this recognition system is quite robust and fast, working well in dynamic natural environment.Then we move to the tracking and localization algorithm, analyze the possibility and suitability of the combination of recognition and tracking algorithm, and illustrate the idea and implementation details of the parallel computing structure. In the following, we introduce the pose estimation algorithm, named POSIT, and explain the mechanism of the whole localization algorithm. Besides, since the perspective camera model is used to compute the 3D coordinates of features extracted from coplanar landmarks, we also briefly review Zhang's camera calibration method.Finally, a serial of experiments is presented to test and verify the performance of our algorithms. Some of the experiments, say, comparison of Harris-SIFT with its several peers, image retrieval in large image gallery, multiple object recognition in natural environment, illustrate the high robustness, accuracy, and real-time performance of Harris-SIFT based recognition system. In the visual localization experiment, we successfully restored the relative 3D pose of a hand-held camera, which could movie rapidly and arbitrarily in the space. It could be concluded from all these experiments that our visual localization algorithm is quite promising for the application of visual localization with only a single camera.