Deep Learning Based Multi-view Depth Estimation And 3D Reconstruction

3D model is an important form of human perception and understanding of the world.With the rapid development of computer technology,3D reconstruction technology is playing an increasingly important role in various aspects of people's life and production,such as cultural relics protection and virtual reality.Among the various 3D reconstruction technologies,the 3D reconstruction technology based on the multi-view image depth maps is one of the most mainstream 3D reconstruction schemes because of its low equipment cost,high operation flexibility and good reconstruction accuracy.In addition,in recent years,with the rapid development of deep learning technology in the field of computer vision,the application of deep learning technology to 3D reconstruction has become a hot research issue.In this paper,we mainly focus on the 3D reconstruction method based on the multi-view image depth maps.The robust and accurate depth maps based on multi-view images are first estimated with the deep learning technology and finally used to build the 3D point cloud models.Based on the work of Point-MVSNet,this paper proposes an improved method of grouping K-nearest neighbor methods,which not only enlarges the spatial perception field in the regularization process of matching cost,but also effectively improves the efficiency of the regularization process.At the same time,a regularization method of matching cost based on confidence weighting is proposed to further improve the quality of depth map estimation and 3D reconstruction.In addition,for the high-resolution input image,this paper uses the multi-scale refinement strategy to refine the depth map of the image gradually.Aiming at the problem of depth search range selection in depth map refinement,an adaptive depth search range estimation strategy is adopted in this paper,which effectively improves the flexibility and accuracy of the algorithm.Specifically,the algorithm in this paper takes a reference image and several adjacent images as input,and firstly extracts robust multi-scale features for each input image through a weighted shared feature pyramid network.Then,we estimate the initial depth map for the reference image at low resolution through a depth initialization network.After that,based on the initial depth map and camera parameters of the image,we carry out the multi-view feature mapping and augmentation to construct the matching cost.The matching cost is then regularized based the grouping K-nearest neighbor search method and the confidence weighting method.Finally,the depth map of reference image is refined through the regression operation.After the above process is repeated for all images and the depth map of each image is obtained,the 3D point cloud model of the scene target is finally obtained through the filter and fusion technology of the depth map,and the 3D reconstruction is completed.This paper is implemented by the Pytorch and experimented on the large 3D reconstruction dataset DTU for small scenes and the dataset of Tank & Temple for large scenes.Aiming at the accuracy and efficiency of the algorithm,this paper makes quantitative analysis on the estimation quality of depth map,quality of 3D reconstruction,GPU memory cost of the algorithm and running time of the algorithm,and compares the results with the most mainstream 3D reconstruction algorithm based on multi-view images.The results show that our algorithm can achieve accurate depth estimation and generate high-quality 3D point cloud model.Compared with the previous algorithms based on deep learning,the GPU memory cost of our algorithm is lower and the running time is faster,indicating that our algorithm is much more efficient.In addition,the experiment also shows that the proposed algorithm can not only perform 3D reconstruction for small scene targets,but also 3D reconstruction for large scene targets,which reflects the good robustness and generalization of the proposed algorithm.