Research On Visual Odometry Algorithm Based On Panoramic Camera

With the advent of the 5G era,high-clarity,high-rate panoramic live broadcast has been possible.As the collection device of panoramic live broadcast,panoramic cameras are also rapidly popularize.In addition to being used for panoramic live broadcast,panoramic cameras can also perform functions such as 3D reconstruction,spatial roaming,positioning navigation,etc.,and the basis for implementing these functions is that the panoramic camera locates itself and restores the motion trajectory,that is,the visual odometry of the panoramic camera.Compared with the traditional vision sensors,which are suffered from limited field of view,the panoramic camera can collect all-round visual information in the scene at one time,which makes the panoramic camera-based visual odometry no longer limited by the angle of view.However,the structure of the panoramic camera is more complicated than that of the traditional cameras.The existing visual odometry technology cannot be directly applied to the panoramic camera.Therefore,it is necessary to firstly calibrate parameters of the panoramic camera and generalize the traditional visual odometry algorithm to be applicable for the panoramic camera.This paper reseaches the calibration and visual positioning of the panoramic camera,and proposes a global optimization-based panoramic camera calibration algorithm and a new visual odometry scheme for the panoramic camera,which enables panoramic cameras to be used in visual positioning and related fields.Considering that the panoramic camera is a ring-shaped rigid structure composed of multi-fisheye lenses,this paper proposes a global optimization-based panoramic camera calibration algorithm.This paper first reseaches the imaging model of the fisheye lens and calibrates the internal parameters of the lens.On this basis,this paper proposes a calibration algorithm for the relative pose between the lenses of the panoramic camera.The algorithm is based on the global reprojection error model and the loopback constraint structure.The relative pose between the lenses is calibrated by optimizing the objective function.The experimental results show that the global optimization-based panoramic camera calibration algorithm effectively improves the calibration accuracy of each lens compared to the accuracy before optimization.The re-projection error decreases by 24.94% on average with the highest drop of 26.53%.The error decreased by 46.24% on average,with the highest drop of 52.86%.The camera parameters of the panoramic camera are obtained by calibration algorithm,so the visual odometry can be realized by using the panoramic camera as a visual sensor.Considering that the panoramic camera is composed of multiple lenses,compared to the traditional camera,the panoramic camera provides more constrains,this paper proposes a visual odometry method suitable for panoramic cameras.Firstly,this paper proposes a PnP algorithm based on histogram statistics to estimate the motion trajectory of a fisheye lens.At the same time,considering that each lens follows the constraint of camera motion,this paper restores the high-precision camera motion trajectory by bundle adjustment method.Experimental results from indoor and outdoor scenes show that the panoramic camera-based visual odometry fits the camera's true trajectory well.