A Coarse-to-fine Estimation Of Spatial Layout Of Indoor Scenes

The estimation of spatial layout of indoor scenes is one of the research hotspots in image processing and computer vision,which is widely used in emerging fields such as 3D reconstruction and scene recovery,etc.There exist always the furniture and other clutters in indoor scenes which will provide relatively rich context clues for the task of scene semantic classification,and result in the borders of the geometric layout of the room to be occluded,while making the estimation task of layout of indoor scenes with only the RGB image become challenging.This paper design a coarse-to-fine estimation approach of spatial layout of indoor scenes whose overall framework consists of scene layout candidates generation based on improved estimation of room vanishing points,rough selection of layout candidates with the global features and spatial layout estimation using regional level features.This paper proposes an improved and ordered vanishing points estimation technique for the indoor scenes in order to solve the problem of being difficult to estimate the vanishing points in the unified order by the traditional methods.First,exploit line segment detection method with local discontinuous and adaptive thresholds to obtain a number of long straight lines from indoor images and divide them into the vertical lines and horizontal ones in terms of the direction of the given line segments.Then,the vote scoring mechanism is used to estimate the vertical vanishing point by calculating the contribution of the lines to the possible vertical vanishing point from the corresponding vertical direction lines.Once the vertical vanishing point is computed,the horizontal and projected direction vanishing points are estimated in terms of the orthogonality principle and the vote scoring mechanism from the corresponding horizontal lines so that the coordinates of the given vanishing points are obtained orderly.Compared with the traditional vanishing point estimation method,the presented estimation method can not only improve the computation efficiency and calculation accuracy,but implement effectively ordered vanishing points estimation.This paper presents a rough selection strategy of indoor spatial layout candidates using the global features,while the traditional method only involves the regional features extraction and learning.First,the corresponding layout boundary information of the given indoor scene is obtained by exploiting the fully convolution neural network(FCN)based on VGG-16 prototype with trained parameters from the input of whole original image,and the Softmax classifier is used to obtain the category of the layout by handling the extracted fc7 layer features from the given FCN network.Then,fuse the obtained boundary and the corresponding category of indoor scenes to generate the global features to constrain the potential boundary of the spatial layout of the whole given scene and roughly select the spatial layout candidates,while reducing the search space of the subsequent layout candidates in favor of efficient selection of potential spatial layout candidates.The experiments on the Hedau and LSUN datasets show that the presented rough selection strategy can achieve the overall constraint of the structure of spatial layout and the location of room boundary of indoor scene.In view of the traditional method with the regional features only from the appearance features such as color and texture,this paper gives a new spatial layout estimation method with optimized selection of the candidate candidates which introduce the geometric regional features from depth and normal information,First,the three-dimensional box spatial layout is parameterized by the angles between the rays from vanishing points to construct the spatial layout estimation model,and the task of layout estimation can be transformed into structured prediction problem.Then,the surface normals and geometric depths of the given scene are estimated by modified spatial multi-scale VGG-based convolution neural network model and transformed into corresponding features.The geometric integral technique is adopted to obtain the regional features of the assumed spatial layout candidates by merging the extracted spatial geometric features and the appearance features from each regional polygon in the given layout.Finally,the Cutting-plane structured optimization method is applied to solve the model parameters.The obtained parameters of the layout model are exploited to estimate the spatial layout of the given indoor scenes by structured prediction with maximization score criterion.The experiments on Hedau and LSUN datasets show that the presented layout estimation can improve the accuracy of the spatial layout estimation,and the adopted geometric cues can be in favor of improving the ability of distinguishing the blocked layout boundary by indoor objects with the presented layout model.