Research On Visual SLAM Based On Multi-Fisheye Camera System

After more than 30 years of development,simultaneous localization and mapping(SLAM)technology is considered as the key technology to achieve autonomous localization of intelligent agents.SLAM technology can be mainly divided into laser SLAM with lidar as sensor and visual SLAM with camera as sensor.Because of the advantages of low cost,large amount of information and wide range of application,visual SLAM has gradually become a research hotspot.The mainstream visual SLAM methods usually use monocular cameras,binocular cameras and RGBD cameras as sensors,but these cameras have limited fields of view and are prone to problems such as tracking loss and occlusion in practical applications.Compared with these cameras,using fisheye cameras or combining multiple cameras can significantly expand the field of view,avoid the above problems as much as possible,and provide richer environmental information for visual SLAM to estimate camera pose and build map.This paper proposes a visual SLAM algorithm based on a multi-fisheye camera system,which makes full use of the large field of view of fisheye camera and the advantages of multicamera system,improves the accuracy and robustness of system positioning and mapping,and solves the problem of scale uncertainty of a single camera.Firstly,in view of the serious distortion of fisheye image,this paper uses Taylor polynomials to model fisheye camera,and proposes a double-pinhole projection model to eliminate image distortion,while retaining image information as much as possible.Then,in response to the problem that mainstream visual SLAM methods do not support multi-camera systems,this paper introduces a multi-camera system model to fuse information from multiple cameras,and proposes a new multi-view bundle adjustment method and its Jacobian formula based on the characteristics of multi-views and a double-pinhole projection model.Subsequently,this paper designs a visual SLAM algorithm based on multi-fisheye camera system,proposes an initialization method suitable for multi-eye systems in combination with the principle of binocular vision,and extends the functions of tracking,mapping and loop closing for multi-eye systems.Finally,some challenging real image datasets are collected in an underground parking lot by using the car equipped with multiple fisheye cameras,and several experiments such as feature matching experiment,visual odometer experiment,closed-loop experiment and camera quantity experiment are designed to evaluate the visual SLAM algorithm based on the multifisheye camera system proposed in this paper.The experimental results show that the double-pinhole projection model proposed in this paper can effectively eliminate fisheye image distortion,improve the quantity and quality of feature matching and increase the average interior point rate of feature matching.The visual SLAM algorithm based on the multi-fisheye camera system proposed in this paper due to the advantages of the double-pinhole projection model and the multi-fisheye camera system has higher accuracy,better robustness and stability,and is easier to detect the presence of loop,thereby further eliminates the cumulative error through loop-closing.