Research On Automatic Aerial Triangulation And Dense Image Matching For Oblique Multi-view Imagery

As a new technology developed in recent years,oblique photogrammetry plays a great role in the rapid acquisition of digital three-dimensional model for large-scale scene,and is widely used in the survey and mapping,government and public data acquisition,urban land managerment,and disaster emergency management,etc.During the post-processing of oblique imagery,the bundle adjustment(BA)and multi-view stereo(MVS)technology are two key steps,which directly affect the automaticity and quality of the digital surveying and mapping results.For the BA problem of oblique multi-view imagery,the classic feature matching algorithms are not accessible due to large oblique angle and rotation angle,scale and light changes,and occulusion.Then,the oblique automatic tie-points detection methods need to by studied.On the other hand,the high-precision POS device is always equipped on large airplane,while the popular UAV systems usually equip with low-precision consumer-level POS device.Therefore,the BA problem of oblique imagery with low-precision POS data or without POS data needs to be studied.Although a great progress has been made for the MVS problem,however,the precision,integrity and reliability are still aspects which needs to be improved.Currently,most of the dense image matching algorithms often fail in regions with sharp discontinuities,weak textures or repeated textures like tall buildings,and areas of vegetation.Those are considered difficult areas in dense matching which needs to be solved.Furthermore,although multi-threads,GPU and other parallel acceleration strategies are adopted in the current commercial software,the efficiency is still a bottleneck in the processing of high-resolution oblique images.Therefore,more efficient MVS algorithms needs to be researched.This paper focuses on the problem of BA and MVS problem for high-resolution oblique imagery,the involved key theories,processing algorithms and applications are researched and the major works are listed follows:1)The uniform imaging mdel framework for photogrammetry(PH)and computer vision(CV)is researched.The geometric three-dimensional reconstruction is widely studied in both PH and CV fields,while the coordinate system definition,mathematical model expression are different due to the background,development and application.In this paper,the two expressions of the collinearity equation are proved to be consistent.Besides,the strict transformation relationship between the interior and exterior parameters is deduced,so that the excellent research results in PH and CV fields can be absorbed and incorporated in to our study.2)The POS assisted automatic tie-points extraction for bundle adjustment of large-scale oblique multi-view images is researched.Firstly,exterior orientation data gained by POS are used to rectify oblique images and predict image correspondences,which can be matched using SIFT algorithm.Secondly,an unordered feature tracking method relies on Union-Find algorithm are adapted to detect the multi-view correspondences.After tie-points extracted,bundle adjustment are done with POS and control points data treated as weighted measurement.In the experiment part,three kinds of oblique images,taken by assembled lightweight oblique system,UltraCam and SWDC-5,are used to test the algorithm above.The results show that our method can adapt the popular oblique systems at home and abroad,which can process over 2000 images at a time.And the precision of oblique triangulation is better than the traditional vertical triangulation,with an accuracy of 0.75 GSD in horizontal direction,2.0 GSD in elevation direction.3)The automatic tie-points extraction for bundle adjustment of large-scale oblique images using on-demand SIFT algorithm without POS data is researched.Firstly,the depth maps and normal maps are reconstructed using PatchMatch-OSSIM algorithm based on the incremental SFM results of low-resolution images with same direction.Then,the candidate principal planes are detected and used to rectify the oblique images followed by SIFT feature detection,matching and tracking.Three kinds of oblique images are used in the experiments,and results show that the proposed method is efficient and valid with an accuracy less than one pixel for the average reprojection error.Compared with ASIFT algorithm,the proposed method is more efficient since only rectification of the principal planes is performed.Moreover,the matching accurate rate is higher than ASIFT due to the reason that lots of useless features are eliminated during the feature extraction.4)The multi-view dense matching algorithm of high-resolution oblique images based on graph newwork is researched.The overlap ratio and intersection angle between image pairs are used to find candidate stereo pairs and build the graph network.A Coarse-to-Fine strategy based on an improved Semi-Global Matching algorithm is applied for disparity computation across stereo pairs.Based on the constructed graph,point clouds of base views are generated by triangulating all connected image nodes,followed by a fusion process with the average reprojection error as a priority measure.The proposed method was successfully applied in experiments on aerial image test dataset provided by the ISPRS of Vaihingen,Germany and an oblique nadir image block of Zurich,Switzerland,using three kinds of matching configurations.The proposed method was compared to other state-of-art methods,SURE and PhotoScan.The results demonstrate that the proposed method delivers matches at higher completeness,efficiency,and accuracy than the other methods tested;the RMS for average reprojection error reached the sub pixel level and the actual positioning deviation was better than 1.5 GSD.5)A novel object-based MVS algorithm using structural similarity index(SSIM)matching cost in a coarse-to-fine workflow is presented.As far as we know,this is the first time SSIM is introduced to calculate the matching cost of MVS applications.In contrast to classical stereo methods,the proposed OSSIM method only computes a depth map for each image.Thus,the efficiency can be greatly improved when the overlap between images is large.To obtain the optimized depth map,the winner-take-all and semi-global matching strategies are implemented.Moreover,an object-based multi-view consistency checking strategy is also proposed to eliminate wrong matches and perform pixelwise view selection.The proposed method was successfully applied on a close-range Fountain-P11 dataset provided by EPFL and aerial datasets of Vaihingen and Zurich by the ISPRS.Experimental results demonstrate that the proposed method can deliver matches at high completeness and accuracy.For the Vaihingen dataset,the correctness and completeness rate were 71.12%and 95.99%with a RMSE of 2.8 GSD.For the Foutain-P11 dataset,the proposed method outperformed the other existing methods with the ratio of pixels less than 2 cm.Extensive comparison using Zurich dataset show that it can derive results comparable to the state-of-art software(PhotoScan,Pix4d and Smart3D)in urban buildings areas.Based on the above research,the automatic bundle adjustment and dense matching software Mogas is developed,which can support and oblique platform and any camera.Besides,a control point selection module is also developed to support interactive manual processing,and POS data and control points are taken as weighted measurements during the self-calibration bundle adjustment.Experimental result show that Mogas software can adapt to the popular oblique multi-camera systems,and the number of processing images can reach more than 2000.