Research On Multi-view 3D Reconstruction Algorithm Based On Deep Learning

Multi-view 3D reconstruction refers to the generation of three-dimensional point clouds from multiple two-dimensional pictures of a given scene taken from different camera positions.Three-dimensional reconstruction is widely used in auto-driving,medical beauty,virtual reality,map mapping,cultural relics protection,military survey and other fields.Multi-view 3D reconstruction has the advantages of low cost and high practicability.Deep learning-based multi-view 3D reconstruction also has problems such as lower output depth map resolution than input resolution,expensive volume regularization,and large memory consumption.In this paper,multiscale image feature fusion,regularization of matching cost volume,and multistage depth information estimation are analyzed and studied.The specific research contents are as follows:(1)In order to solve the problem that the resolution of the output depth map is lower than the input resolution,a method for extracting and fusing multi-scale image features is proposed.The fusion of multi-scale image features makes full use of the spatial context information between different scales of the input image,improves the integrity of the reconstructed object,and outputs and inputs depth maps with the same resolution.Experiments show that the simultaneous use of multi-scale image features can improve the integrity of the reconstruction.(2)To solve the problem that cost volume regularization consumes a lot of memory,this paper presents a regularization method combining cyclic neural network and U-Net network model to divide the cost volume into depth planes and regularize them.Experiments show that the proposed method can reduce the memory consumption during reconstruction and can reconstruct scenes with larger resolution.(3)A multi-stage depth estimation method using image pyramids is proposed to improve the accuracy of reconstruction by optimizing the depth information in multiple stages.The validity of different regularization methods in different depth estimation stages is verified.The training and performance evaluation of the network on DTU datasets have significantly improved the overall performance.