Oblique Photogrammetry Based Scene 3D Reconstruction With Structure Sensing Functions

Oblique photogrammetry based 3D reconstruction is one of the research hotspots in the field of photogrammetry and remote sensing.Compared with the traditional vertical aerial images,the oblique image has the characteristics of short base line,large geometric deformation,large color difference,serious ground-object occlusion and large amount of data,which brings a series of difficulties and problems to the full automatic high-precision 3D modeling for oblique images.This paper focus on the research of scene full-automatic 3D reconstruction based on oblique photogrammetry.The current oblique photogrammetry based automatic 3D reconstruction methods have many problems,such as they cannot process all kinds of oblique images automatically,the model structure is not prominent and the precision of the 3D model is not high.In order to deal with these problems,we have done a series of in-depth research work and given some solutions for every stage of the oblique photogrammetry based 3D reconstruction.1.Overview of oblique photogrammetry.The characteristics of oblique images are introduced and summarized.The imaging methods of oblique images are reclassified and systematically summarized.According to my classification theory for the imaging methods of oblique images,some typical oblique photography platforms are introduced.2.At the oblique image matching stage,as the large geometric distortion of oblique images makes the automatic oblique image matching is difficult or low efficiency,we designed two oblique image matching methods.(1)A multi-source or multi-group large-view image matching method without POS data is proposed.We supposed that the length of photography baseline is known(or estimated).For multi sources or multi views images,images at the same view are matched and orientation separately,and we got the 3D object points of the corresponding image points at the same view.For each view,we used the 3D object points to build patches,and used the PMVS method to refine and verify the patches to remove the affine distortion of the local texture.We used the center of the patches as the feature points and used the SIFT descriptor to descript them.Then,for multi views,we matched these feature points from multi views images and the matching result can convert to multi views image matching result.(2)For the oblique images with rough POS data,an automatic view-point invariant image matching method for oblique images is proposed.The matching points obtained by this method are numerous,distributed evenly and have high accuracy.3.At the stage of bundle adjustment for multi-views oblique images,in order to reduce the amount of unknown adjustment parameters(overmuch unknown adjustment parameter may weaken the instability of adjustment solution),we offered a new bundle adjustment model for oblique images which took the relative attitude parameters of cameras into account,and also gave the application scope of the model.4.At the stage of dense matching for oblique images,we introduce a Pyramid strategy based semi-global dense matching and depth map fusion method.In order to address the following problems:(1)The amount of UAV image data is very large,but ordinary computer memory is limited;(2)the patch-based multi-view stereo matching algorithm(PMVS)does not work well for narrow-baseline cases,and its computing efficiency is relatively low,and thus,it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed.This paper proposes an Image Grouping and Self-Adaptive Patch-based Multi-View Stereo matching algorithm(IG-SAPMVS)considering the terrain structures for multiple UAV images.First,multiple UAV images were grouped reasonably by a certain grouping strategy.Second,image dense matching was performed in each group and included three processes.(1)Initial feature matching,which consisted of two steps:The first was feature point detection and matching,which made some improvements to PMVS,according to the characteristics of UAV imagery.The second was edge point detection and matching,which aimed to control matching propagation during the expansion process.(2)Self-adaptive patch matching propagation considering the terrain structures.Initial patches were built that were centered by the obtained 3D seed points,and these were repeatedly expanded.The patches were prevented from crossing the discontinuous terrain by using the edge constraint,and the extent size and shape of the patches could automatically adapt to the terrain relief.(3)Filtering the erroneous matching points.Taken the overlap problem between each group of 3D dense point clouds into account,the matching results were merged into a whole.Experiments conducted on several sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without compute memory restrictions,the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.5.At the stage of 3D meshing and texture mapping,aiming at the problems that the Mesh model is not smooth and the edges and corners are not prominent,we designed a new high-precision Poisson surface reconstruction method by considering the plane structures and boundary features.And we designed an automatic high-resolution texture mapping method with occlusion detection functions and got the 3D model with real textures.Experiments showed that the precision of the model is high and the visual effect is good.The methods in this dissertation are applied to the 3D reconstruction software"Photo3D" developed by Wuhan University.At the last,the feasibility of real-time image processing and 3D reconstruction is discussed.By integrating the key technologies of UAV photogrammetry and some visual SLAM technologies,a feasible scheme for real-time UAV image aero-triangulation and DOM generation is proposed,which provides a correct solution for real-time image processing of UAV.