A Data Driven Approach To Multiview 3D Reconstruction

3D reconstruction is always a promising research topic in area of computer vision and computer graphics, the goal of it is how to reconstruct 3D model for special scene.Nowadays, we are live the era that the technology of virtual reality (VR) and augment reality (AR) are developing with many requirements. Owing to 3D model is one of the most important basis for VR and AR, thus researchers must be faced to the problem that how to reconstruct the 3D model with high precsion as soon as possible. Hower,traditional 3D modeling method such as manmade is difficult to provide a 3D model with high reality and the process lacks interactivity. Moreover, the traditional modeling method is evenly not suitable for the big scene. There are many cheap cameras at present and easily to capture a plent of videos and images,whereas the computer vision technique can provide a computational model which is correspondence to the perceptual law of hunmanity. These can make up the deficiency of the traditional modeling method. As a result, the computer vision based 3D modeling technique is mainly developing tredency.Based on above background, this dissertation focus on the data-driven technique for 3D reconstructon,which includes how to obtain the camera intrinsic information and extrinsic parameters, and 3D geometry of the scene from the collection of video or image.Moreover, this dissertation has solved the problem of lighting and geometry consistent way,and could provide a perfect solotion about 3D modeling for many practical applications. In summary, the main comtributions of this dissertation as follows:(1) We propose a fast and robust feature tracking framework. In order to efficientl hand image rotation,occlusion and repeating feature,we propose a feature point detector with main directon and feature descriptor which has strong representiveness. Based on the proposed feature point detector and descriptor,we propose a matching method for multiple views, this method can produce accurate matches without additional computational cost. Thus, high-quality feature tracking method cloud directly push the development of struction from motion technique and enhance the key basis of 3D reconstruction with large-scale scene.(2) We propose a fast and robust method to estimate the absolute camera pose. It is the first work to formulate the problem of absolute camera pose estimation in null-space of the vector which obtained from camera center to 3D point, as a result this method only requires six 2D-3D correspondences to compute the absolute pose information.Moreover, to further improve the precision of the estimated absolute camera pose,we propose a non-linear least squares method to refine the initial estimated pose to obtain a high-precision camera pose information, and this is very helpful in triangulation and bundle adjustment.(3) We propose a fast and robust bundle adjustment method,which is very suitable for large-scale 3D reconstruction. This is the first work to down-weight the potential outliers using robust loss function. To reduce requirement of memory space for bundle adjustment in large-scale scene, we split the large-scale bundle adjustment problem to many smller ones according to the sparsity between camera and 3D point,so that these smaller ones can be solved easily with small memory space. Moreover,we propose to use compressed column storage (CCS) technique to store spare matrix,this could futher improve the use ratio of memory. Finally, we make use of the intrinsic property of Jacobian matrix to compute the unknown paramers using a fast matrix factorization method, and result in a low computational cost. To further speed up the proposed bundle adjustment method, we have implemented the multiply operation of high dimensional matrix to save computation time. As a result, an excellent bundle adjustment method could help to reconstruct 3D model with high geometry consistent way..(4) We propose a novel framework for multiview based 3D reconstruction. The proposed framework is composed of structure from motion (SFM), multiview stereo (MVS)and surface reconstruction. Firstly,we propose a novel SFM pipeline,initial view.selection method and saliency based disambiguation method to improve the quality of 3D point-clouds produced by SFM. Secondly, we propose a general MVS (GMVS)with automatic initial view selection to produce high-quality and dense 3D point-clouds. Thirdly, we propose to use the multi-band blending technique to refine the reconstrusted texture, and essensure the final mesh has high geometry and illumination consistent way with the real scene. Finally, based on the innovation of theory and practice, we have developed an automatic 3D system named AutoX3D,which could push the development and application of theory about 3D reconstruction.