Multi-view Stereo Vision Method Based On Deep Learning

In the field of computer vision and computer graphics,three-dimensional reconstruction technology has significant research importance and has been widely applied in various fields of life such as automatic driving,restoration of ancient cultural relics,virtual reality,and others.Among the 3D reconstruction techniques,the multi-view stereo vision(MVS)method is a classic method in computer vision.Traditional MVS methods relied on the stereoscopic relationship between multiple images to match costs and worked well in strong texture areas or ideal Langber scenes.However,matching difficulty in weak texture areas and non-diffuse reflection areas due to a lack of information resulted in poor reconstruction results.With the rapid development of deep learning technology in computer vision,the multi-view stereo vision method based on deep learning technology has effectively improved the reconstruction effect by independently learning a large amount of information.However,challenges still exist,and this paper aims to conduct an in-depth analysis of 3D reconstruction technology and study the multi-view stereo vision method based on deep learning technology to further improve the reconstruction effect of the 3D model.(1)In this paper,we propose a multi-view reconstruction method based on gradient and gaussian process regression that aims to address the issues of feature maps being sensitive to lighting changes and incomplete reconstruction caused by insufficient use of information between images in the MVS method.Firstly,a feature extraction network with a fusion gradient is designed,which improves the influence of gradient information in the feature map and enhances the inhibitory force of the feature map on the influence of lighting change factors.Secondly,we introduce a view feature enhancement module that integrates the Gaussian process regression method to address the limitation of the feature extraction step in multi-view reconstruction,which only focuses on the current view without considering the potential spatial relationship between views.This module effectively increases the impact of relevant information between views on the multi-view stereo vision reconstruction task and improves the completeness of the reconstruction results.Finally,since the Gaussian process regression algorithm makes the feature volume distorted by the feature map add the information between different perspectives,the contribution of different views to the Cost Volume is calculated to construct a Cost Volume that is more in line with visual perception.Our method significantly improves the reconstruction effect compared to advanced multi-view stereo vision reconstruction methods on a public dataset.(2)In this paper,we propose a semantic-enhanced multi-view stereo vision method that aims to address the issue of deep convolutional neural networks lacking low-level semantic information in their feature extraction.Firstly,we propose a Conv LSTM(Convolutional Long Short-Term Memory)semantic aggregation network that uses the Conv LSTM network structure to predict the feature map extracted by multiple convolutional layers.This approach results in a feature map that integrates the semantic information of each layer,allowing us to extract high-level features layer by layer in space.The long short-term memory neural network structure’s memory function enhances the low-level semantic information in the high-level feature map,leading to improved reconstruction in weak texture regions and greater robustness and integrity in 3D reconstruction.Secondly,we propose a visibility network that highlights the visible area’s characteristics on the feature map and deepens the visible area’s influence,resulting in better three-dimensional reconstruction.Finally,this method is tested on the public DTU dataset and Tanks and Temples dataset,and compared with the mainstream multi-view stereo vision reconstruction method,the reconstruction effect is better.