3D Reconstruction Based On Image Sequences

Information that human being perceive is mainly gotten through vision.Computer vision aims at to duplicate the effect of human vision byelectronically perceiving and understanding images, reconstruct the 3D scene,then complete the task under the specific environment in special duty. 3Dreconstruction is always one of the hottest directions in computer vision field.It is the technology to research how to get 3D information of object from the2D information of object.This thesis systematically addresses 3D reconstruction technology basedon image sequences: from camera calibration, feature extracting, featurematching, to epipolar geometry, the fundamental matrix and the essentialmatrix, then completes the 3D reconstruction of two images;afterward wetake the 3D reconstruction results of two images as a foundation,accomplished feature tracking, bundle adjustment and 3D reconstruction ofimage sequence. Also we present our experimental results --the spot cloudsstructure which showed by VRML, the experimental results are satisfying.Based on above, we discuss as follows.1. Feature extracting and feature matchingThis paper presents a new interest point descriptors representationmethod based on independent components analysis (ICA). The aim of thisalgorithm is to find a meaningful image subspace and more compactdescriptors. Combination the descriptors with an effective interest pointdetector, the proposed algorithm has a more accurate matching rate besidesthe robustness towards image deformations. The proposed algorithm firstfinds the characteristic scale and the location for the interest points usingHarris-Laplacian interest point detector. We use Haar wavelet transform onthe neighborhood of the interest points and get low frequency gradient featurevectors. Then ICA is used to model the subspace and reduces the dimensionof the feature vectors. The experiments shows that ICA based method is moreaccurate and more fast than SIFT method.2. Epipolar geometry and fundamental matrixUnder the identical world coordinate system, images of the one objecthave a restraint relations in geometry,we called it epipolar geometry. Instereo vision, we can recover this constraint by feature matching, on thecontrary, also we can use this geometry to constrain feature matching, whichcauses the hunting zone for corresponding points reduce from the 2D plane tothe epipolar line, and enables the matching robustness and precision all toobtain a very big enhancement.Epipolar geometry can be described by a 3×3 matrix of rank-2. It isindependent of the scene structure, and only depends on the cameras' internalparameters and relative pose. The matrix is called fundamental matrix.Therefore, the epipolar geometry problem transforms to the fundamentalmatrix F estimate question. Precisely calculates fundamental matrix has thevital significance to calibration, nicety matching and 3D reconstruction. Thisthesis introduced sevaral solutions of the fundamental matrix F : seven-pointalgorithm, eight-point algorithm, and emphatically introduced the robustRANSAC algorithm and present satisfying experimental results.3. Essential matrix EIf we know the camera intrinsic matrix K, we may obtain the essentialmatrix E directly by the fundamental matrix F. The essential matrixcontained external parameters information of the camera. This thesisintroduced how to solve the camera's projection matrix P by the essentialmatrix E and the camera intrinsic matrix K in detail: (1) decomposes fourgroups parameter R and t by the essential matrix, then get four groupsprojection matrix P (2) determines the correct P using the depth of fieldcomputation.4. The reconstruction of two imagesIf a certain space dot M is observed by a camcorder, its image dot is m.Because the image dot of any dot M ' laid the line CM is m, the position ofthe dot M is not determined only by dot m. The space dot M is observed bytwo camcorders C and C ', two image dots is dot m and dot m ', if we canconfirmed that dot m and dot m ' are all the image dots of the dot M, andthen its 3D position is determined. Because the dot M is not only lied lineCM , but also located line C 'M , the dot M is the intersection of twobeelines CM and C 'M . Namely its 3D position is may the onlydetermination. This is the basic principle of 3D reconstruction in stereovision.We present the algorithm of 3D reconstruction of two images as follows:(1) Obtains the camera intrinsic matrix with the traditional calibrationmethod.(2) Extracts features of two images, carries on feature matching.(3) Acquires the fundamental matrix F using RANSAC algorithm basedon feature matching results.(4) Obtains the essential matrix E by the fundamental matrix F.(5) According to the essential matrix E, gets the correct group ofextrinsic camera parameters R and t.(6) Combines extrinsic camera parameters R and t to the projectionmatrix P1 and P2 .(7) Reconstructs the matched features of two images.(8) Applies bundle adjustment to the reconstruction result of two images.Finally, we present the reconstruction experimental results of twoimages. Through the results, we might clearly see that we could get 3Dinformation of objects by two related images.5. 3D reconstruction of image sequenceWe use KLT algorithm tracking features, obtains the features trackingtable, then reconstructs the first two images of the sequence (with two imagesreconstruction method) to get an initial structure. Then take one image of thesurplus images to join the initial structure, reconstructs the new added imageby feature tracking relations. When reconstructs the new image added, weused the robust RANSAC algorithm to get the camera's projection matrix Pi .After again we introduced bundle adjustment--an important technology in 3Dreconstruction.The bundle adjustment optimizes the reconstruction result by to provide"whole optimized" of the 3D structure and the camera parameter. Here"optimized" refers to the final parameter estimation to be possible to causesome price function to obtain the minimum value;"Whole" then meant itssolution gets optimization in spite to the 3D structure or to camera parameter."Bundle" indicated that the light group of lines leaves from the objects in 3Dto be able to collect by each camera center. The bundle adjustment enhancesthe estimated value's precision through unceasing optimizing data.We present the 3D reconstruction algorithm of image sequence asfollows:1. Obtains the feature tracking table by using KLT algorithm to track thewhole image sequence.2. Reconstructs the first two images of the image sequence by twoimages reconstruction method.3. Refines the reconstruction result of two images through bundleadjustment.4. For every additional view:(1) According to corresponding relations of the 3D spots which alreadyreconstructed and the new added image's features, computes thecamera projection matrix P using the robust algorithm RANSAC.(2) According to the projection matrix of the current image, removesthese features which have big error.(3) Reconstructs the features of current image and the former, whichhas not been reconstructed.(4) Removes 3D spots which have big error reconstructed by step (3).(5) Refines all present images through bundle adjustment.5. Finally makes a bundle adjustment for whole image sequence.Finally we have presented the experimental results with analysis. Weobtain sparse 3D space spots from image sequence reconstructed, and useVRML to show its spots clouds structure. We can observe the object structureclearly. We also presented the reconstruction error under each kind ofparameter, as well as the analysis of these situations.