Online Reconstruction And Understanding For Indoor Scenes

The indoor scene is the centre environment of human activities.The research of indoor scenes has always been one of the most attractive topics in computer vision and computer graphics.And the online scene reconstruction and understanding are particularly widely studied on account of their ability of in-time feedback.With the blooming of depth sensors and data-driven methods,the last decade witness a rapid development of online scene reconstruction and understanding.However,to satisfy the emerging demands in practical application,the robustness and accuracy of the online algorithm are remained to be improved.For the online indoor reconstruction,the existing online scene reconstruction methods are limited to slow and smooth camera motions.Under the fast camera motion,the severe motion blur and large inter-frame camera motions would boost the difficulties in camera tracking,lead to a significant drop in algorithm performance.And for the online scene understanding,it’s challenging to maintain the dynamic geometry information along with the scanning.Most of the existing consider only texture information,ignoring the crucial contribution of geometry information.To address the above-mentioned issues,this thesis proposes some novel solutions,which significantly improve the robustness of the online reconstruction method and the accuracy of the online understanding method.Firstly,we merely leverage the depth information to build the fitness function.And for the first,we propose a non-gradient method to estimate the camera pose.We also designed an efficient dynamic data stucture,which can organize the time-varying geometric information for 3D point convolution,which improve the accuracy of online scene understanding.The main contributions include:Real-time dense scene reconstuction based on random optimization: we estimate the camera motion by sampling lots of candidate camera motions in the solution space.Compared with commonly used linear approximation methods,random optimization is more adaptable to highly non-linear optimization.Meanwhile,a fitness function that measures the conformance of the depth information and TSDF is designed to avoid the motion blur problems.Experiments show that our methods work stably under fast camera motions(4m/s).Online semantic sengmentation with global-local tree: The geometric information of scenes can contribute to a more precise understanding.We propose a novel data structure called the global-local tree,which is composed of a global tree and per-point local tree.The global tree and local tree are respectively designed for partitioning the 3D space and maintaining the local surface information.With the global-local tree,the method can dynamically organize the time-varying geometric information and combine with convolution network for semantic prediction.The results from the public dataset demonstrate our method improve 19.1% of accuray than the best online semantic segmentation methods.In the end,by combining the online reconstruction with online semantic understanding,an online perception system is proposed for future complex and high-level applications.