Research On Computation And Quality Evaluation Method For Large-Scale Multi-View 3D Reconstruction

The goal of multi-view 3D reconstruction is to recover 3D models of objects from multi-view images.The technique has a wide range of applications in heritage protection,geographic mapping,urban planning and other fields.With the development and popularization of mobile cameras,it is becoming more and more convenient to capture photos,and the scale of scenes,the number of photos,and the complexity of scenes in reconstruction problems are also increasing.On the classic Middlebury benchmark datasets taken in the laboratory environment,the multi-view 3D reconstruction methods can approach laser scanning in terms of accuracy and completeness.However,as the reconstruction scale increases,the multi-view 3D reconstruction methods face new challenges,including:due to the high computational complexity,the reconstruction can hardly run on a single machine for large-scale scenes,and the application of the divide-and-conquer strategy can avoid the overall reconstruction,which breaks the upper limit of the computing power of a single machine,but the dividing and merging process of existing approaches cannot guarantee the quality of the reconstruction;in situations with weak textures,occlusions,etc.,the reconstructed models have local defects or missing parts,and solution to these problems still needs further study;for quantitative evaluation of camera parameters and 3D geometric reconstruction results,there is still a lack of suitable benchmark datasets.In this thesis,we study the problem of large-scale multi-view 3D reconstruction from two aspects,including reconstruction methods and quality evaluation.Specifically,the work and innovations of this thesis include:· For the bundle adjustment step,which is the global optimization step of the reconstruction process,an overlapping parameter fusion method based on asynchronous consensus is proposed,which optimizes the sub-problems and fuses the consensus alternately and iteratively,and reduces the inconsistency between sub-problems through the transfer of overlapping parameters.With an asynchronous way,the waiting time of the child nodes can be reduced and the calculation efficiency can be improved.Also,this method can be proved to obtain the optimal solution of the original problem.A block partitioning method based on scene summarization is also proposed.It improves the partitioning efficiency by merging similar scene points and cameras,and selects overlapping parameters to optimize uniformity.The experimental results on the BAL public datasets and CH datasets show that this method can reduce the calculation overhead through asynchronous methods and scene summarization,thereby improving the solving efficiency.· A data-driven local reconstruction repairing method is proposed.First,point cloud completion is performed using the reference model library.Based on local 3D descriptors this method uses a cascading way to quickly filter out unrelated 3D models.Then,during the process of fine alignment,the reference model is smoothly deformed to achieve seamless completion.After acquiring the approximate shape of the missing area,we propose to utilize photometric consistency constraints to improve the quality of the local images,thereby improving the reconstruction quality.Fusing with the original model,the geometry and texture of the missing area can be restored.In experiments on the Model Net40 dataset,we prove that the point cloud completion method can efficiently make use of a large 3D model set to improve the quality of the target point cloud.The experimental analysis of multi-view 3D reconstruction cases prove the effectiveness of the local reconstruction and repair method.· In large-scale multi-view 3D reconstruction,the existing benchmark datasets are limited in terms of the scale of scenes and the accuracy of ground-truths.In this thesis,an approach to establish a benchmark dataset based on synthetic data is proposed.It uses images captured in a real scene and the corresponding 3D model to simulate the photo shooting process to obtain accurate ground-truths and a large number of reconstructed images.With the one-to-one correspondence between original images and synthetic images,some properties such as the image shooting matrix and distortion are transferred to make synthetic images as real as possible,and verify these images according to the similarity to their corresponding real photos.Using the proposed method,a large-scale multi-view 3D reconstruction benchmark dataset,Yungang Grotto,is established,and the main steps in the reconstruction pipeline are quantitatively compared.· The existing distributed computing framework cannot fully consider the characteristics of the multi-view 3D reconstruction problem.In this thesis,a distributed computing framework based on pickle is proposed and named as Pickle RPC.It encodes instructions and data into byte streams and transmits then between nodes to achieve efficient remote procedure calls.On the basis of the master-slave computing nodes,an additional control node is added for backup of image data and intermediate computation results in case of master node failures,making the top-level reconstruction algorithm run efficiently and stably.This thesis combines reconstruction algorithms,the distributed computing framework and post-processing methods to build a distributed multi-view 3D reconstruction prototype system.