The Key Technology Of Visual/Inertial/UWB Integrated Positioning System

With the development of industry and economy,mobile robots play a critical role in the human society and have a wide range of real-world applications.The accurate localization in an unknown environment serves as the fundamental building block for mobile robot to navigate autonomously and complete various vision tasks.Therefore,Supported by the National Natural Science Foundation of China,the research in this paper focuses on the key technology of visual/inertial/UWB integrated positioning system,and aims to solve the problem of mobile robot autonomous localization in the complex unknown environment without using satellite signal.In this paper,a visual/inertial/UWB integrated positioning system is designed completely from hardware to software level.Furthermore,the system is capable of operating in different modes according to varying environments,and thus can provide a precise and robust localization information for mobile robots.The main work of this paper is as follows:1.The prototype of the visual/inertial/UWB integrated positioning system is designed.Firstly,we choose the suitable visual-inertial,UWB sensors and a powerful central processor to design a complete hardware system with the robot chassis.Secondly,based on the ROS and its communication mechanism,we design the mobile robot software platform with drivers between sensors and central processor.Finally,a complete visual/inertial/UWB integrated algorithm with different modes suitable for different environments is designed based on the hardware and software system.2.The visual-inertial positioning system based on points and lines is explored.The pointbased visual or visual-inertial positioning system suffers from the difficulty of extracting and matching feature points in the dim,low texture or structured environment.To alleviate this problem,we introduce the line features complementary to the point features.Furthermore,we have designed a complete visual-inertial system based on points and lines with IMU preintegration and graph optimization methods.Meanwhile,we utilize the keyframe selection,sliding window and state marginalization approaches to suppress the computational cost and improve the real-time performance of our system with high accuracy.3.The loop closure with uniform landmark sampling method is proposed.The performance of existing loop closure methods demonstrates the declined performance in case of large environment and viewpoint variations.To handle this problem,we introduce the image patches and develop the landmark-based loop closure scheme.Firstly,we conduct a comprehensive study in which the influence of image patches on the loop closure performance is explored,and thus derive two-fold important observations in terms of the beneficial effect of the specific landmark generation strategies.Inspired by the above observations,a simple yet effective uniform sampling based scheme is presented for landmark generation and accurate loop closure detection in this paper.The experimental results on the four challenging datasets demonstrate that the loop closure performance can be significantly improved by our efficient approach in which the landmarks are appropriately produced for accurate pairwise matching.4.The camera re-localization scheme using landmark and keypoint matching is proposed.Camera re-localization is a challenging task,especially based on the sparse 3D map or keyframes.In this paper,based on the landmark-based loop closure detection method with uniform sampling,we present an accurate method for RGB camera re-localization in the case of a very sparse 3D map built by limited keyframes.Our approach is essentially a top-to-down feature matching strategy to provide a set of accurate 2D-to-3D matches.Specifically,we first use the landmark-based loop closure detection method to generate from keyframes the images similar to the current view along with the set of pairwise matched landmarks.Then,the points are matched within the landmark pairs and combined afterward.Finally,we utilize this scheme for the re-localization task and conduct experiments on the challenging datasets.Experimental results demonstrate that the camera poses estimated by our method based on the sparse 3D point cloud are more accurate than the classical methods using the dense map or a large number of training images.5.The UWB localization method and the visual-inertial,UWB data fusion approach is designed.Firstly,we combine the visual-inertial system based on points and lines with the loop closure and camera re-localization scheme to suppress the accumulative errors of the visualinertial system.Then,since the existing visual-inertial positioning system suffers from the accumulative errors and can not achieve global localization in the fixed coordinate system,we have introduced the ultra-wideband sensors and designed optimization-based UWB localization method.Besides,we have presented the data-fusion framework of UWB and visual-inertial system.Specifically,we have utilized the UWB to build the global constraint of the pose and employed the visual-inertial system for building the related constraint.Thus,the UWB and the visual-inertial system are fused effectively.This scheme allows providing the body pose in the global coordinate system and further improving the accuracy and robustness of the positioning system.6.We conduct the experiments to evaluate the prototype of the visual/inertial/UWB integrated positioning system in the real-world environment.In our experiments,we evaluate the performance of visual/inertial/UWB integrated positioning system in the scenarios of ideal environment,the environment with varying illuminations and the low-texture environment.In addition,the situation of sensors failure is also evaluated.The experiments results show that our system can alternate between different environment-specific modes smoothly,and provide a stable and high-precision pose information in a complex unknown environment.