The Research On The Omnidirectional Vision System For Autonomous Mobile Robots

On the basis of designing and realizing the omnidirectional vision system for autonomous mobile robots, the thesis focuses on how to improve the robustness of the omnidirectional vision system, and how to achieve robust object recognition and self-localization based on omnidirectional vision for autonomous mobile robots.The autonomy of mobile robots depends on the ability to percept the environment, and visual sensors can provide the richest environment information for autonomous mobile robots. Among all of the visual sensors, the omnidirectional vision system can provide a 360o view of the robot's surrounding environment in a single panoramic im-age, and the robot can use it to realize object recognition, mapping, self-localization, etc. by image processing, analyzing and understanding. So it has become more and more popular as a visual sensor for autonomous mobile robots. Although a lot of progresses have been achieved in the applications of omnidirectional vision for autonomous mobile robots, many challenges still exist because of the complexity and the difficulty of this problem. These challenges include not only the common problems in computer/robot vision research, but also the problems brought forth by the special imaging character of omnidirectional vision. For example, how to achieve the constancy in the image acqui-sition, processing and understanding of the robot vision is still challenging under dy-namic lighting conditions; the computation costs of the existing local visual feature al-gorithms used by robots when working in the complex and unstructured environment are usually high, which limits the actual application of local visual features in the engi-neering situations with high real-time requirements; the local visual feature algorithms should be adjusted according to the special imaging character of omnidirectional vision; the ability to recognize the ordinary objects based on omnidirectional vision should be improved to reduce the constraints of the robot's working environment; the robot's self-localization should be robust with respect to the highly dynamic factors in the in-door and structured environment; more research results in the pattern recognition com-munity should be introduced and applied to the robot's self-localization in the unstruc-tured environment, so the robot can cognize the environment and realize self-localization in a more consistent way with human; etc..The following research is performed and finished in the thesis to deal with the challenges mentioned above:(1) The design and calibration of the omnidirectional vision system. The design and calibration of the catadioptric omnidirectional vision system are summarized com-prehensively in the thesis, and then a novel omnidirectional vision system named as NuBot is designed and realized by using RoboCup Middle Size League (MSL) soccer robots as the test-bed, which also provides an example on how to design the omnidirec- tional vision system. Because the NuBot omnidirectional vision system is not a single viewpoint one, a model-free calibration idea is adopted to perform the calibration of the distance map for the system.(2) Camera parameters auto-adjusting technique based on image entropy. The varying lighting conditions will affect the images acquired by vision systems, which will cause much difficulty to the robot's object recognition and self-localization. In the thesis, the definition of image entropy is presented, and then it is verified that image en-tropy can indicate whether camera parameters are set properly by experiments. A cam-era parameters auto-adjusting algorithm based on image entropy is proposed to achieve some kind of color constancy in the output of vision systems, so the robustness to the varying lighting conditions can be improved for the robot's vision system. The algo-rithm is tested by using the NuBot omnidirectional vision system and the perspective camera in indoor and outdoor environments, and the results show that the algorithm is effective.(3) Two novel real-time local visual features for omnidirectional vision. To deal with that local visual features can not be extracted in real-time by the existing algo-rithms, two novel real-time local visual features, namely FAST+LBP and FAST+CSLBP, are proposed in the thesis for omnidirectional vision to make local vis-ual features applicable in the actual engineering problems with high real-time require-ments. The matching experiments of the panoramic images from the COLD database are performed to determine their optimal parameters, and to compare with the famous SIFT in performance and computation cost. The experimental results show that the proposed local visual features perform better, and can be extracted in real-time.(4) Arbitrary FIFA ball recognition based on omnidirectional vision for soccer ro-bots. The conclusion is derived that the ball is imaged to be the ellipse in the panoramic image acquired by the NuBot omnidirectional vision system by analyzing the imaging character, and then an image processing algorithm is designed to detect the arbitrary FIFA ball according to this imaging character. A ball velocity estimating algorithm is also integrated to track the ball in real-time. The recognition process is independent on the color information, so the robustness of the robot's omnidirectional vision can be improved, and the current color-coded environment of RoboCup MSL can be further changed to promote the realization of the final goal of RoboCup. The experimental re-sults validate the effectiveness of the algorithm.(5) Robot's robust self-localization based on omnidirectional vision in the highly dynamic indoor and structured environment. Taking the RoboCup MSL competition as the application background and the test-bed, a novel self-localization method based on omnidirectional vision is proposed in the thesis. The method combines the particle filter localization and the matching optimization localization, which are two most popular ones currently, by utilizing their virtue and making up their respective deficiency. So the highly accurate self-localization can be achieved in real-time while global localiza-tion is realized effectively. By integrating the camera parameters auto-adjusting algo-rithm based on image entropy, the robot's self-localization is robust to the highly dy-namic environment with occlusions, high speed competing between robots and varying lighting conditions. The experimental results in the standard RoboCup MSL field show that the algorithm is effective.(6) Robot's topological self-localization based on omnidirectional vision in the un-structured environment. In the unstructured environment, the visual features applied in the robot's navigation and localization can not be understood by human, and the robot's topological self-localization based on feature matching directly needs huge memory space, which is inconsistent with the way human cognize the environment. The bag of features method, which is popular and has been applied successfully in pattern recogni-tion problems, is introduced to the robot's self-localization in the thesis. By combing the two real-time local visual features proposed in this thesis and Support Vector Machines (SVM) classifier learning algorithm based on statistical learning, a place recognition algorithm based on bag of local visual features and SVM is presented for mobile robots, meanwhile the robot's topological self-localization is also realized by place recognition. The COLD database is used to perform the experiments to determine the best algorithm parameters and training conditions, and the results verify the effectiveness of the algo-rithm.