Sparse Point Cloud Reconstruction Based On Image Sequence

The three-dimensional sparse point cloud reconstruction technology based on monocular vision is an important application of computer vision and image processing in engineering.It provides an effective solution for reconstruction of three-dimensional scenes from image sequences or video.Therefore,it is of great practical significance to study several problems in three-dimensional sparse point cloud reconstruction.This paper aims at reconstructing the sparse point cloud from the image sequence and discusses the key issues in the reconstruction process such as camera calibration,feature extraction and matching,triangulation,and bundle adjustment.The main work of this paper is as follows:(1)Realize camera's non-linear calibration.Do monocular camera calibration based on camera nonlinear model.The nonlinear model is converted into two parts of linear and nonlinear parameter equations.The singular value decomposition is carried out for the coefficient matrix of the linear part of the parameter equations after the direct linear transformation,and the optimal estimation of the linear partial parameters is obtained.Then,the nonlinear optimization method is applied to the nonlinear parameter equations to fit the nonlinear parameters,so as to achieve camera calibration.(2)Realize the detection and matching of feature points.The SIFT algorithm with scale invariance and affine invariance is used to extract the feature points,and the initial matching is completed according to the proportional test method.Then apply the RANSAC normalized eight-point method,filter the interior points and calculate the best estimate of the fundamental matrix.(3)Through semantic segmentation,RANSAC normalized eight point method is improved in the plane scene.The eight-point method has many degenerate cases.For the most common case in which eight points lie in a same plane,Mask R-CNN is introduced to perform detection and semantic on matching images,with the semantic segmentation information,the degenerate sampling subsets are eliminated during the RANSAC algorithm.The algorithm effectively improves the robustness of the eight-point method in this degenerate scene.(4)Realize the bundle adjustment method and complete the three-dimensional sparse point cloud reconstruction process.Taking the re-projection error as the loss function,the bundle adjustment method is used to optimize the parameters in this incremental reconstruction process.Through triangulation,camera registration and projection matrix calculation,bundle adjustment,the reconstruction of three-dimensional sparse point cloud is achieved.