3D Point Cloud Reconstruction From A Single Image

The provided information from a single image is limited,most traditional methods(e.g.stereovision,structure from motion)need two or more images to recover the 3D shape of the object.But those methods usually have wide pipeline and cannot reconstruct objects lacked texture.Currently,with the popularity of mobile devices,a single image can be seen everrywhere.Therefore,3D reconstruction from a single image has great practical effects and economic benefits.In the past ten years,deep learning technology and the development of large-scale repositories of 3D CAD models provides the necessary conditions for the 3D reconstruction from a single image using data-driven methods.However,recently-proposed methods require the objects be captured with relatively clean backgrounds,fixed viewpoint,while this highly limits its application in the real environment.This paper proposes a novel solution to reconstruct the 3D point cloud from a real single image.The main work and innovations of this paper include:(1)For 3D model retrieval problems,a 3D model retrieval method based on rendered images is proposed.Firstly each 3D model in the pre-prepared 3D model database is rendered from different viewpoints to build a large-scale rendered image dataset,and then the VGGNet-16 network is employed for feature extraction.Finally,a feature map database dictionary is built for all 3D objects.When a query image appears,the nearest object model is obtained by comparing the features extracted from the query image with the feature map stored in the database dictionary;(2)For real images in complex backgrounds with varnous view angles,this paper propose using prior-knowledge for 3D reconstruction from a single real image.We design an end-to-end network RealPoint3D,which can take both the 2D image and 3D point cloud together for 3D object reconstruction.Different with the existing works,RealPoint3D can reconstruct objects with from an image with complex background and changing of viewpoints;The proposed method is evaluated on images from ShapeNet dataset and ObjectNet3D dataset.Experimental results show the state-of-the-art performance on the synthetic rendered and real images,compared with other generation methods.Furthermore,the proposed framework works well for real images in complex backgrounds with various view angles.