The Research Of 3D Reconstruction From A Single RGB Image Based On Deep Learning

The last few years have witnessed a continued interest in 3D reconstruction based on single-view RGB images.The performance of 3D reconstruction has improved dramatically,due to the great success in RGB image-based deep neural networks.However,the outputs of existing learning models are mostly volumetric representations of 3D shapes.In other words,these deep models map RGB images to 3D images(voxels are similar to pixels).Even if the quality of 3D volumetric representation is high,it does lose some important information about the 3D shapes,such as the shape topology and the relations between the parts.In order to recover the structure of the 3D model with more completed and detailed information,we propose a 3D reconstruction method based on a single-view RGB image,which is a convolutional-recursive auto-encoder consisting of a process of parsing the structure of a RGB image and a process of recovering the cuboid hierarchy.Firstly,given a single RGB image,the structure mask network extracts the contour information and structure features of the interesting object in the RGB image.Secondly,the structure reconstruction network decodes the features and uses the cuboid and tree hierarchy to represent each part and the relations between the parts respectively.The relations include connection relations and symmetry relations(the symmetry relations include connection,symmetry and parallelism,etc.).Finally,the thesis realizes the process of automatically recovering the cuboid representation of the parts of the target object and the part relations.The purpose of the structure mask network is to parse the RGB image.It is a multiscale convolutional neural network that can be used to estimate the contour information and structural features of the target object at various scales and environments.The goal of the structural reconstruction network is to decode the RGB image features to obtain the structural information of the 3D shape.The decoder fuses the features extracted by the mask network and the features of the original image,then recursively decodes into the hierarchy structure of the cuboid.Since the decoding network can recover the connectivity and symmetry between the parts of the 3D shape,the deep learning model we proposed can ensure the rationality and versatility of the reconstructed 3D shape.The deep learning model we proposed is jointly trained by contour-mask and cubestructure training data.During the training stage,several mechanisms are devised to avoid over-fitting.Through the experimental results of this thesis,we can see that the method we proposed has achieved very successful results.The method of this thesis recovers the detailed 3D shape structure of the target object from the RGB image with very high quality.After comparing with other state-of-the-art work,it fully demonstrates that the method we proposed is full of innovation and value.There are two applications of this thesis including structure-guided completion of 3D volumes recovered from a single RGB image and the structure-aware interactive editing of RGB images.