Multi-View Image Based 3D Reconstruction Considering Structural Information In Urban Scenes

Image-based three-dimension reconstruction has been one of the most important fields in photogrammetry and computer vision.As an important source data for detecting the threedimensional shape information of the object surface,the image has unparalleled advantages in terms of economy,convenience,flexibility when compared with other ways(such as light detection and ranging and microwave interferometry).The increasing resolution of photography and the wide range of applications of oblique photogrammetry make it possible to carry out fine spatial information comprehension of urban scenes.And the high-quality three-dimension reconstruction is an important prerequisite for this.For urban scenes,there are still many challenges in achieving of accurate three-dimensional reconstruction with complete structural information.The apparent fronto-parallel and a large amount of unevenness in the reconstruction results also limits the application of the results.There are a large number of planar structures,sharp edges and corners in the urban scene.These obvious geometric features bring challenges to 3D reconstruction,while provide more abundant constraints that could be used to improve the reconstruction performance in urban scenes.However,classic 3D reconstruction algorithms do not provide good technical practice and theoretical support for these constraints.To address this problem,this thesis takes the multi-view images based three-dimensional reconstruction of urban scene as the basic problem and proposes a reconstruction method that takes into account the unique geometric structure of urban scenes such as planes and sharp edges.By using the relationship between two-dimensional images and three-dimensional spatial information,the objective function is constructed by using multiple condition constraints.The global optimal solution obtains high-precision estimation of three-dimensional information in urban scenes and achieves accurate expression of shape structure,efficiently.The specific research work is as follows:This thesis proposes a planarity constrained multi-view depth(PMVD)map reconstruction method.Using a planar geometry model,which describe the relationship between an image pixel and its corresponding local plane in the Euclidean space,as the local shape presentation,PMVD is able to reconstruct the depth and normal(vector)of the object simultaneously.The reconstruction starts with image segmentation and feature matching to initialize the local planes as optimization labels.Then,it utilizes an iterative optimization scheme,where each iteration is composed of candidate label selection,belief propagation,and outlier elimination.Based on the segmentation structure,PMVD extends the Patch Match method to update the candidate labels at the beginning of each iteration.Finally,multiple criteria are used to eliminate image matching outliers.(Vertical)Aerial images,oblique(aerial)images and ground images are used for qualitative and quantitative evaluations.The experiments demonstrated that the PMVD outperforms the popular multi-view depth map reconstruction with an accuracy two times better for the aerial datasets and achieves an outcome comparable to the state-of-the-art for ground images.For all of the tested datasets,as expected,PMVD preserved the planarity for piecewise flat structures in urban scenes and restored the edges in the depth discontinuous areas such as roof-wall junctions.Line segments and corner points have the good ability to describe the geometry of objects in urban scenes.The plane structure in the urban scene is enclosed by line segments and corners.In this thesis,This thesis studies the three-dimension reconstruction method,with use of multiple geometric structures such as line segment,corner point and plane,based on the image triangulation.Firstly,a density-adaptive image triangulation is constructed by using line segments and corner points obtained by feature extraction.That means,there are a large number of triangles in the rich texture area,and the number of triangles in the lack of texture is small.Then,based on the image similarity measure and smoothness constraint,we constructed the energy function by adding continuity constraints on the edges between adjacent triangle.Finally,the EM-like optimization framework is used to minimize the energy,which could determine the optimal object plane for each image triangle.In the process of optimization,the image fracture line is determined by graph cut,thereby improving the performance of the reconstruction result at the edge of the image.Qualitative experiments show that the proposed method can obtain the reconstruction results of the geometric structure better.Quantitative experiments show that the reconstruction accuracy of the three-dimensional information is better than the current mainstream algorithms.For the fusion of three-dimension reconstruction results,including the geometrical structure information,generated by images,the strategy of directly averaging the depth information or the normal vector will seriously destroy the original geometric structure information.In this thesis,we explored how to fuse this kind of reconstruction results and propose a fusion method to generate outlines.Firstly,the image contour feature is extracted from the image by using the planar structure constraint.Then,we construct a cross structure and determine the corresponding relationship of the structure in different images;Finally,the image contour line is fused by the cross structure,and the object outlines composed of threedimensional line segment capable of effectively expression of the three-dimension information and accurately describing the geometric collision of the object are obtained.