Research On Multi-fisheye Camera System In Visual SLAM

The robust estimation of the camera pose lies at the core of many image based applications in robotics,photogrammetry and computer vision.In recent years the focus is put more and more on the real-time capability,generality and the possibility to use the developed methods in complex,large-scale indoor and outdoor environments.Especially in challenging indoor scenarios,the field of view of the cameras plays an important role and it is often increased by using fisheye lenses or camera mirror combinations.Apart from increasing the field of view using fisheye lenses,multiple cameras can be rigidly coupled and combined to form a multi-camera system.In the case of multi-camera systems the redundant observations can be effectively used to ensure a robust pose estimation and observed object features are longer visible.Therefore,a multi-fisheye camera system for Simultaneous Localization and Mapping(SLAM)can capture richer environmental information for more robust pose tracking.In this paper,the multi-fisheye camera system is applied to the visual SLAM algorithm,and the following aspects of work are carried out around the key problems encountered:(1)An feature descriptor-based intrinsic calibration method for fisheye cameras is proposed.The feature descriptor-based calibration pattern with a high number of detectable features is used for camera calibration by using a polynomial model that accurately describes the physical properties of any kind of camera lens.The residual function is replaced and all parameters are weighted jointly refinement in the calibration process.The experimental average error is 0.3053,the root mean square value(RMS)is 0.3752,and the mean square error(MSE)is 0.3085,the calibration results meet the requirements of calibration accuracy.(2)A relative pose estimation method based on mapping method is proposed.The easily exploited slam mapping-based method is employed to calibrate the relative poses of cameras using the improved feature description calibration pattern,and the method is equally applicable to non-overlapping multi-camera systems.All the cameras are tracked in the same map,and each camera's pose is initialized with the same calibration pattern.The calibration results can be applied to a multi-camera SLAM system to verify the accuracy of the relative pose calculation.(3)An improved feature extraction algorithm for fisheye camera is implemented.The method uses the AGAST detector instead of the FAST detector to improve the efficiency of feature detection.At the same time,the method uses the improved dBRIEF descriptor to directly describe the distortion feature,and the distortion image is not required to be corrected as a whole,but only the binary sequence is distorted,so that the descriptor is adapted to different image regions.The experimental results show that the improved algorithm can be directly applied to fisheye images,and the matching accuracy is greatly improved compared with the original algorithm.(4)A multi-camera visual model that can cope with complex environments is implemented.By introducing a virtual point,the observations of each camera at the same time in the multi-camera system are combined into one observation equation,and the observation results of each camera can be separated.The experimental results show that the multi-fisheye camera system using the multi-camera visual model can operate stably for a long time in complex environments such as strong exposure and rare texture,and pose tracking has strong robustness.