Research On 3D Semantic Scene Completion Method Based On Generation Adversarial Network

Scene 3D reconstruction technology is an important research topic in computer vision and positioning and navigation.Using this technology can obtain the 3D contour of the object,as well as the coordinates of any point on the contour.With the continuous development of visual navigation technology,people are more and more interested in using computers to simulate real-world 3D scenes,and the requirements for 3D reconstruction technology are getting higher and higher.3D reconstruction based on a single image is an important research direction in 3D reconstruction.The main difficulty in the reconstruction process is insufficient information.However,due to its high reconstruction efficiency and low cost,it is used in virtual reality,large-scale scene reconstruction,and urban digitization.And cultural relics are widely used.The purpose of semantic segmentation is to divide objects into regions and to label the objects in each region semantically.The purpose of 3D reconstruction is to recover the 3D spatial information of objects or scenes based on 2D images.Although semantic segmentation and3 D reconstruction are two different research areas,there are actually many connections between the two.This article considers transforming these two issues into the same task,and completes the completion of RGB-D images to 3D scenes with semantic information through a designed adversarial network.The main contributions of this paper are as follows: 1.A 3D semantic scene reconstruction method based on generative adversarial networks is proposed.Aiming at the problem of poor accuracy of complex scene reconstruction in existing convolutional neural network methods,we propose a novel method to complete semantic scene reconstruction from a single depth map;2.Aiming at the problem that the reconstruction effect of 3D semantic scenes under different data sets is greatly different,an end-to-end deep learning framework is proposed to complete the RGB-Depth completion of the depth channel of the D image;3.In order to solve the problem that existing SSC networks rely on depth maps without considering RGB images with more comprehensive information,an RGB-based D double stream 3D semantic scene reconstruction method,which can effectively reduce the parameters in the network and save running time.At the same time,based on the existing RGB-D dual stream network,a network structure optimization method is proposed.By introducing an encoder of voxel data,a discriminator of reconstruction results,and a discriminator of latent features,the depth and color image features are seamlessly merged,so that the network can infer more fine-grained semantic information from the RGB images.In this paper,two public datasets SUNCG and NYU are used as training and testing samples for training the RGBD-SSCGAN network.Compared with the benchmark method,the accuracy of the proposed method is improved by 4.4 % and 3.2 % on the scene scenario task(SC)and three-dimensional semantic scene reconstruction task(SSC).In summary,the method proposed in this paper has good accuracy for SC and SSC targets achieved by different input sources,which proves that the method described in this paper is effective and robust.