Panoramic 3D Reconstruction Based On Multi-camera System

Panoramic 3D reconstruction is a 3D scene reconstruction method using panorama sequence captured by panoramic photography, which is an important content of the 3D reconstruction theory. Due to the scene to be reconstructed can be covered by relatively small number of panoramic images, panoramic 3D reconstruction can effectively simplify the complexity of 3D reconstruction and improve its efficiency, especially for large-scale 3D scene reconstruction.In this paper, some relevant theories and methods of panoramic 3D reconstruction were deeply researched, and a 3D structure recovery algorithm based on spherical camera model was designed. Thereby, the panoramic sparse point cloud reconstruction was accomplished using panoramic image sequence captured by multi-camera system.Firstly, according to the features of the panoramic image captured by the multi-camera system, the spherical camera model was used to represent its projective relationship. Then, the method of building spherical camera model from a multi-camera system and the representing and storing of the spherical panoramic image were discussed.Secondly, the spherical panoramic image feature extraction and matching algorithm was researched in this paper. Combining with the SIFT algorithm and the spherical image processing method, a spherical SIFT algorithm was proposed, which can realize robust feature extraction and matching for spherical panoramic image.Thirdly, the multi-view geometry under the spherical camera model was researched and the two-view and three-view geometric under the spherical camera model were deduced. Meanwhile, some concepts, such as the spherical epipolar geometry, the spherical essential matrix, the spherical homography, the spherical trifocal tensor etc., and the calculation of spherical essential matrix and structure under the spherical camera model were disscussed. Furthermore, this paper introduced the estimation of spherical epipolar error and spherical reprojection error.Finally, a panoramic 3D structure recovery algorithm based on the spherical camera model was fully designed in this paper. Thereinto, some methods based on spherical camera model were discussed, such as the spherical camera matrix recovery and parameter optimization. The spherical camera matrix recovery and the panoramic sparse point cloud reconstruction were therefore fulfilled by the spherical epipolar geometry and spherical bundle adjustment etc. methods with the feature tracking information obtained by using spherical SIFT algorithm.