Research On 3D Reconstruction Of Target Based On Video Sequence

3D reconstruction is one of the key research directions in the field of computer vision in recent years,and it plays a vital role in smart home,virtual reality,cultural relic reconstruction and other aspects.The cost of 3D reconstruction directly using color images is lower than that of 3D reconstruction by depth cameras such as Kinect,so it is meaningful to study 3D reconstruction based on video sequence images in this paper.The traditional 3D reconstruction method must manually select the features for stereo matching,which requires a high demand on the scene and is difficult to extract the local features of the image,so the accuracy of 3D reconstruction is often not high.Deep learning methods have been widely used in the field of computer vision because they can extract more extensive and accurate features,and 3D reconstruction is also included naturally.The depth map is one of the four forms of 3D reconstruction.Since the quality of reconstruction depends on the depth information of the image,the main research content of this paper is how to accurately estimate the depth information of the image.In this paper,the scene of video sequence image acquisition is divided into two global motion states and local motion states of the camera.Therefore,the paper mainly carries out the following two aspects:(1)For the global motion state of the camera when the target scene or object is still(mostly indoor scene),this paper proposes an improved MVSNET network depth estimation method based on unsupervised learning,which uses deep learning network to replace the traditional feature matching to improve the integrity and accuracy of the 3D reconstruction results.In this paper,pyramid feature aggregation module is used to extract image features and group normalization is adopted to extract image features better.The GRU structure with improved parameters was used to perform regularization and classification operations to estimate the initial depth of the input image.A joint bilateral filter with similarity factor of depth value was used to process the initial depth map.The training of the network adopts the robust luminosity loss formula for unsupervised learning.After verification and analysis on the DTU data set,compared with the most advanced algorithm,the algorithm in this paper improves the accuracy of the 3D reconstruction results,and the average integrity reach 0.525 respectively.(2)For the local motion state of the camera including moving objects(mostly outdoor scenes),this paper also designs a depth estimation method based on the motion model based on unsupervised learning,the method of two-dimensional images on a single object movement to explicit modeling,using a network to estimate motion model 3 d motion vector of a single object in the scene,to generate the target object in the state of the target frame;Using Packnet architecture instead of the traditional depth estimation network to learn the mapping from color image to each pixel depth map;The reprojection error loss function is used as one of the constraints of the network to train the unlabeled monocular video.Through training and testing on the KITTI data set,the method in this paper is superior to other methods in estimating the corresponding depth of the image when there are moving objects in the video sequence,and the threshold accuracy of the depth map reaches 0.880.To sum up,this paper focuses on the research of 3D reconstruction of target based on video sequence.Two ideas are designed according to the two states of the camera to obtain the video sequence,and these two algorithms are verified and discussed respectively and improved and optimized in the experimental process.Experimental results show that the reconstruction results of the proposed method are more complete and the depth estimation results are more accurate.