Layout Estimation Of Indoor Scenes Based On Conditional Generation Adversarial Networks

Image scene understanding has been the research hotspot in computer vision,indoor layout estimation is the key of the task of scene understanding.For the problem of layout understanding of indoor scenes with RGB images,this paper designed a novel method that is based on conditional generative adversarial networks(CGAN)for indoor layout estimation.First,to deal with the blur of the edge maps caused by interpolation,the depth of convolutional layers and deconvolutional layers are increased.An encoder-decoder network is proposed to construct the generator of the GAN to produce edge estimates that has the same size of the input images.Second,to deal with the difficulty of convergence during training the GAN,this paper introduces to employ multi-scale supervision to construct the generator with an extra output branch so as to boost the convergence of the training.Then,based on the original input image and the corresponding ground truth edge map,the generator is built with the encoder-decoder structure and multi-scale supervision and the discriminator is built with a convolutional neural network,which mutually construct the model for indoor layout estimation.Guided by the adversarial loss,the adversarial networks are trained using mini-batch Stochastic gradient descent.High-resolution layout edge estimates are generated based the networks.Later,the vanishing points are estimated with traditional method,from the three vanishing points a seies of rays are uniformly generated to make sectors.The candidate sectors are selected for maximizing the average edge strength.Gaussian blur is employed to the high-resolution edge estimates,in order to expand the coverage of the edges and generate a series of high-quality layout hypotheses.At last,the optimal layout estimate is selected which maximize the convergence area and similarity of the layout hypotheses and the generated high-resolution layout edge estimates.Experimental results and analyses on the two benchmark datasets LSUN and Hedau show that,compared to other layout estimation methods,the proposed method can have the global perspective to understand the indoor scene layout.Hence the proposed method can accurately predict the 3D spatial structure and segment the semantic surfaces,and improve the accuracy of layout estimation.In addition,the proposed network for indoor layout estimation have fine generalization ability and might be used in all kinds of complicated indoor scenes.