Research On 3D Reconstruction Of Multi-View Wide Baseline Images Based On Point And Line Features

With the fast development of computer and information technologies, there is more and more demand for 3D object or scene models in our everyday life. Comparing with traditional 3D modeling software and 3D scanner, 3D reconstruction based on multiple images has following characters:●The cost of hardware is low, and only simple device, such as hand-held camera, is needed.●The operation is easy because more and more tasks can be done by computer, which leads to less and less human operation than 3D modeling software and scanners.●We can gain 3D models of photorealism, which is difficult for 3D modeling software.Technologies involved in 3D reconstruction have promiseful importance both in theory and in practice, are widely used in virtual reality, object recognition, image indexing, untouched measuring, telecine, animation making and so on. In light of above reasons, 3D reconstruction based on multiple images becomes a hot research topic in computer vision and computer graphics fields in recent years.The aim of this thesis is to obtain 3D model and structure of real scenes from multi-view wide baseline images based on point and line features, and put our algorithms, methods and ideas into practice. The key technologies include: feature point matching for wide baseline images, projective reconstruction, camera calibration, Euclidean reconstruction, generation of 3D models, and acquisition of textures. The thesis is focused on the above associated issues, employing the knowledge from image processing, computer graphics, computer vision, pattern recognition, optimization theory, etc.This dissertation focuses on the reconstruction problem of geometries and textures from multiple wide baseline images, some creative algorithms and methods have been proposed, and the highlighted ideas and main contributions are described as follows:●An accurate wide baseline matching method was proposed, with large geometric and illumination deformations betweeen images. There are geometrical deformations between corresponding feature windows on different images of the same patch on scene surface, which can be represented by a homography (projective transformation) on 2D plane, but most of the state-of-the-art matching methods use feature descriptors which are invariant under special cases of this group. In this part, an overview of invariant features under different geometric groups was given first, and then a quantificational analysis of affine invadant feature was proposed. The analytic expression of location error was deduced in theory by our novel method, and the feasibility was demonstrated by means of experiments. At the same time, some valuable conclusions and suggestions for wide baseline image matching based on above analysis were proposed. Based on the discussions of affine invadant features, an accurate wide baseline image matching method was proposed based on our by-stage affine iteration scheme, which uses a similar transformation model followed by an affine transformation model using our fast iterative optimization algorithm. After getting the optimal affine parameters, not only the location errors of corresponding feature points were compensated, but the deformations of corresponding feature windows were rectified. Results and comparisions of public real testing images were given, which strongly demonstrate the feasibility and accuracy our method.●An accurate 3D Euclidean reconstruction method for points was proposed, based on approximate internal parameters of digital cameras. Theoretical analysis of projective deformations between traditional projective reconstruction and Euclidean reconstruction based on stratified projective geometry was given first, and then some suggestions which can relieve these deformations were deduced. An accurate 3D Euclidean reconstruction method from images without knowing exact internal parameters of cameras, named with 3D quasi-Euclidean reconstruction was proposed, and an accurate linear triangulation algorithm under the quais-Euclidean frame was given. Comparing with traditional stratified reconstruction strategy, we estimate the initial intrinsic parameters using the technical manual of digital cameras, avoiding the complex and cumbersome self-calibration process. Strictly speaking, our initial result is projective reconstruction, but it is very close to perfect Euclidean reconstruction. We get the accurate global optimal reconstruction based on bundle adjustment across image sequence. Synthetic data and real scene images demonstrate the feasibility and accuracy of our method.●A 3D reconstruction method of line features extracted from multi-view images was proposed. Compared with traditional feature point based reconstruction method, this thesis uses line features as the basic elements for 3D reconstruction. Our contributions consist of three aspects: first, the average geometric reprojection error was used as the objective function for triangulation and optimization, assuring the global optimization of final results; secondly, an novel analytic correlation method of bilinear constraint for noisy 3D lines represented by Plucker coordinates was proposed, assuring the validity of 3D line coordinates, and both academic proof and geometrical meaning of our method were given; thirdly, a minimal representation for 3D lines with 4 freedoms was used during final bundle adjustment, which overcomes the intrinsic constraint in the 6-parameter of Plucker coordinates, and strongly improves the accuracy of final results. Experiments of synthetic data and real images demonstrate the feasibility and accuracy of our method.●After implementation and integration of above algorithms, a 3D reconstruction system from multi-view wide baseline images based on point and line features was designed and set up. The framework and modules of the system were given first, and then the detailed issues during implementation are settled, which includes the generation of topologic wireframe of 3D models based on discrete points and lines, and automatic extraction and rectification of texture for 3D models. Some 3D scenes reconstructed by this system are shown.●Based on above practice of our 3D reconstruction algorithms, we made another application, that is, augmented reality by computer vision means, which includes registration of 2D virtual objects(label, words, etc.) on projective images of real scene planes, and registration of 3D virtual(graphics) objects on perspective images of arbitrary real scenes. Experiments of real images give an unartificial visual effect, which demonstrates the feasibility and accuracy of our 3D reconstruction algorithms from another point of view. Based on the research work of this thesis, some practices from theory to application were implemented for visual geometry, which promote the combination between computer vision and computer graphics, and build solid ground for other relevant theories and applications.