Visual 3D Reconstruction Using Unstructured Image Collections

With the simplicity of image acquisition and the growing ubiquity of handheld cameras,imagery has become an important medium for scene analysis and shape re-construction.One popular mechanism to extract this 3D information is structure-from-motion(SfM).SfM is an active field in computer vision and graphics community,and can easily recover the 3D geometry(scene structure)of a scene via a set of images from different viewpoints(camera motion).Due to the flexibility and simplicity,it has been widely used in industry and military,such as the digitalization of cultural relics and navigation for drones.Different from recent scanning based methods,which rely on particular scanning machines,the visual based technique(structure from motion)is device-independent.It can be used for reconstructing both small-scale indoor scenes and large-scale landmark architectures.Additionally,rather than merely processing the successive frames in videos,SfM is competent to both structured and unstructured photo collections.While recent advance of SfM makes it more efficient and practical,there still are many significant issues need to be solved.In this dissertation work,we addresses three relevant problems existing in recent structure from motion systems,the acceleration of image collection matching,refinement of feature correspondences and disambiguation of duplicate scenes.We first seek an efficient way to construct match graph for SfM problem in large-scale datasets,then introduce a matching criteria for quantitative and denser feature correspondence,finally present an intuitive solution for ambiguity defi-ciency caused by repetitive structures in human world.Specifically,the main contents and the contributions of our work can be concisely stated as follows:(1)Present a novel and scalable feature-oriented image matching algorithm for large collections.Our method improves the match graph construction procedure in three ways.First,instead of building trees repeatedly,we put the feature points of the input image collection into a single kd-tree and select the leaves as our anchor points.Then we construct an anchor graph from which each feature can intelligently find a small portion of related candidates to match.Finally,we design a new form of adjacency matrix for fast feature similarity measuring,and return all the matches in different photos across the whole dataset directly.Experiments show that the feature-oriented correspondence algorithm can explore visual connectivity between images with significant improvement in speed.(2)Introduce a novel feature matching algorithm for image collections,which is capa-ble of providing quantitative depiction to the plausibility of feature matches.We achieve this by exploring the epipolar consistency between feature points and their potential correspondences,and reformulate feature matching as an optimization problem in which the overall geometric inconsistency across the entire image set ought to be minimized.We derive the solution of the optimization problem in a simple linear iterative manner,where a k-means-type approach is designed to auto-matically generate consistent feature clusters.Experiments show that our method produces precise correspondences on a variety of image sets and retrieves many matches that are subjectively rejected by recent methods.We also demonstrate the usefulness of the framework in structure from motion task for denser point cloud reconstruction.(3)Propose a novel algorithm for SfM disambiguation that explores the global topol-ogy as encoded in photo collections.An important adaptation of this work is to approximate the available imagery using a manifold of viewpoints.We note that,while ambiguous images appear deceptively similar in appearance,they are actually located far apart on geodesics.We establish the manifold by adaptively identify-ing cameras with adjacent viewpoint,and detect ambiguities via a new measure,geodesic consistency.We demonstrate the accuracy and efficiency of the proposed approach on a range of complex ambiguity datasets,even including the challenging scenes without background conflicts.The three main contributions of this dissertation concentrate on the topic of structure from motion,and serve each other in close dependency.Experiments show that the proposed methods in the thesis are effective and scalable,and would be instructive for other problems in computer vision and graphics.