Research Of Room Layout Reconstruction Algorithm Based On Panorama

Room layout reconstruction is an important research topic in the field of computer vision and plays an important role in indoor scene understanding.The layout reconstruction task aims to locate the 3D corner coordinates and reconstruct the 3D room layout.The field of view(FoV)of the panoramic image is 360°,which has the advantage of global context compared with perspective image and can provide complete geometric structure information for layout reconstruction.Recently,great progress has been made in 3D room layout reconstruction by panorama using deep neural networks.However,many current algorithms first locate the wall-floor boundary and wall-ceiling boundary,and then postprocess the corners to reconstruct the room layout,which results in redundant prediction results and the inability to incorporate geometry awareness in the loss function.In addition,most recent studies use bidirectional long short-term memory network(Bi-LSTM)to build room layout geometric information,but Bi-LSTM cannot handle the panoramic feature sequence of circular structure well,resulting in discontinuous prediction results.To solve the above problems,this paper studies geometry-aware room layout reconstruction based on a single panorama.The main research contents are as follows:1.This paper represents the room layout as a sequence of horizon-depth and a room height value for the first time,and proposes an omnidirectional-geometry aware loss function of horizon-depth and room height.Meanwhile,a plane-geometry aware loss function is proposed to further improve the performance.It includes the normal loss and normal of gradient loss to supervise the planeness of walls and turning of corners.2.This paper proposes an efficient deep neural network framework,which consists of a panoramic feature sequence extractor and a panoramic feature sequence processor.The feature sequence processor is based on Transformer,which is composed of(shift)window block and global block to enhance local and global geometry relations.Meanwhile,for the panoramic feature sequence of cyclic structure,a symmetric relative position encoding is proposed to enhance the spatial recognition ability of Transformer in panorama tasks.Furthermore,a post-processing algorithm through occlusion detection is also proposed.3.Experiments on multiple datasets show that the loss function and network architecture proposed in this paper achieve better performance than most of the current advanced algorithms.Ablation experiments verify the effectiveness of each component proposed in this paper,including room layout representation,loss function,network structure,position encoding and post-processing algorithm.