Multi-View 3D Reconstruction Based On Deep Learning

As one of the important technologies of environmental perception,3D reconstruction has been widely used in many fields in recent years,and has attracted more and more attention.Traditional 3D reconstruction algorithms are difficult to solve problems such as lack of scene texture and high computational cost,the use of deep learning methods to deal with these problems is a current research hotspot.This thesis mainly studied the multi-view 3D reconstruction method based on deep learning in multi-view feature extraction,invisible view and reconstructed point cloud accuracy.The following research work was carried out:(1)Optimization of view features extraction network.Based on the multi-view 3D reconstruction algorithm,an improved view feature extraction network algorithm was proposed.In order to reduce the loss of details of the source view after the homography transformation,this algorithm added a two-channel attention mechanism to the algorithm.The multi-view was passed through the channel attention module and the spatial attention module,respectively,to strengthen the acquisition of view features and the positional relationship between multi-views.Experimental results showed that the feature extraction network with dual-channel attention mechanism not only improved the detailed performance of input view features,but also focused the features of different views to the same spatial position,which reduced the integrity error of the reconstructed point cloud by 25.6%.(2)Optimization of view selection network.Based on the equal weight aggregation cost volume model,an adaptive weight cost volume aggregation algorithm was proposed.In order to obtain more local texture features in the reconstructed point cloud,a weight network composed of a three-dimensional convolutional neural network was proposed,so that the multiview feature body after the network was subjected to variance aggregation based on different weights,and the reconstruction ability of the algorithm was strengthened.Secondly,the effective information was enhanced by expanding the number of views at the input end to reduce the interference of useless information.The experimental results showed that the algorithm can obtain more invisible point information when the cost volume is aggregated,and solved the problem of low local integrity caused by the invisible part of the scene.Compared with the point cloud through the two-channel attention mechanism,it further reduced the point cloud integrity error by 12.5%.(3)Newton iteration method to optimize the initial depth map.Based on the method of using neural network to optimize the depth map,a newton iteration method was proposed to optimize the initial depth map.First,a residual function was constructed by mapping the reference view to the space where the source view was located to obtain feature differences.Second,the newton iteration method was used instead of the neural network to obtain a more accurate depth stack.The experimental results showed that the optimization of the initial depth map by the newton iteration method reduced the point cloud accuracy error by 27.1%,and retained a relatively good point cloud integrity to a certain extent,and achieved a more balanced final result.