Automatic 3D Model Reconstruction From Multi-View Images

3D model acquisition is a fundamental issue in the fields of computer graphics and computer vision. However, to construct 3D model manually using software such as 3D MAX and Maya is a tedious and expensive work. Therefore, finding out how to obtain 3D model directly from the real world becomes a hot topic in the fields. Currently, as a digital reservation and record technology, 3D structure acquisition from real objects has a great deal of potential application need in fields of object modeling, scene modeling, photorealistic rendering, robot navigation, object recognition and 3D metrology and in other cultural fields such as archaeology, advertising and entertainment.Methods for 3D model acquisition from real objects can be mainly categorized as active and passive approaches. Active methods are used by 3D scanner community and passive method usually defined by 3D modeling based on images. Due to its low cost and immediately color acquisition power, image-based 3D modeling becomes complement of active methods.Approaches of 3D modeling based on images can be mainly divided into two groups: methods based on calibrated images and ones based on un-calibrated images. Methods based on calibrated images need to put a calibrated target into object scene in advance, which are of time and space limit. While methods based on un-calibrated images only rely on the feature correspondences among different scene views. At present, methods based on un-calibrated images always use image sequences of small baseline, which need too many images to obtain an integrated 3D model.We start from feature correspondences and the deduced geometry constraints among un-calibrated images. Aiming at constructing integrated 3D model, we exploit the whole workflow of 3D reconstruction. The main contributions are the following:1. Propose a new image local feature descriptor.A technique to construct efficient and distinctive descriptors for local image features is presented. We start with the scale invariant features detected and the gradient data of their neighborhood patches in suitable size normalized, and then apply Independent Component Analysis (ICA) to obtain the independent components of the feature patches. Experimental results demonstrate that the proposed local feature descriptor is distinctive and of high matching speed. The descriptor can be used for feature matching in case of wide-baseline multi-views and reduces the number of images used for reconstruction work, which provides a good start for the whole 3D reconstruction task.2. Propose algorithm for retrieving 3D point set based on 2D information.We design and implement the algorithm of the 3D point cloud recovery andcamera motion parameter estimation from 2D image space.(1) We give an effective and robust method for nonlinearly estimating the fundamental matrix using global optimization and bundle adjustment techniques jointly. The fundamental matrix is parameterized to the minimum number of seven parameters and initially estimated by global minimization in term of non-convex linear matrix inequality (LMI) and convex LMI relaxation techniques. We perform the computation in a RANSAC framework and consider nonlinear criteria, together with epipolar geometry constraint.(2) We deduce the multi-view projective reconstruction in a unified framework only based on the fundamental matrix, in which the 3D structure is created. Based on the unified framework and the solved fundamental matrix as well as the incremental technique, the projection matrices in projective space corresponding to different views are estimated. The cumulated errors during the estimation procedure are decreased by strategies of pair-view estimation, triple-view optimization. The solved projective matrices and 3D point set in projective space can be updated to the metric space by self-calibration technique. Due to the robustness and preciseness of the estimated fundamental matrices, our method to recover the projection matrices and 3D structure is stable and robust.3. Propose an algorithm for optimizing the recovered 3D point set.(1) We propose a non-linear 3D point set optimization algorithm based on SBA framework and random walk model. Under the consideration that the feature matching pairs are the input parameters of optimization algorithm, we propose an anisotropy random walk model to resample the correspondences in image space. The refined correspondences and the estimated projection matrices as well as the initial recovered 3D point set are as the parameters being sent to SBA framework to perform local and global optimization. RANSAC framework is also used to increase the robustness of the algorithm.(2) We discuss the possibility of repairing the recovered 3D point set based on the silhouettes of target object in image space. The heart of our approach is to adjust inversely the retrieved 3D points according to the silhouette informationprovide in the sampled views. We start with the set(M|^), which containing the points needed to be repaired, calculated first, and then propose two method toadjust points in (M|^) along the opposite directions of their normals.4. Propose a multi-view texture mapping algorithm based on Matthew Effect probability model.We propose a new method for automatically texturing complex 3D model. The photographic images for acquiring 3D structure are again used as texture data and mapped precisely on the surface to enhance the model appearance. Inspired by law of Matthew Effects, we implement an iteration algorithm for automatically sampling a few images as texture maps instead of using the whole set of input images. In addition, the color difference and discontinuity existing in seams between texture blocks where texture maps belong to different views is relived using multi-texture blending technique and weight schema. Furthermore, a kind of small holes in 2-manifold texture space is filled by a simple weight strategy.