Research And Implementation Of Indoor Scene Real-time Segmentation Algorithm Based On The Depth And Color Data

As the population ages,the development of indoor mobile robots has been promoted the height of national strategy.In order to implement the high level vision tasks,such as object recognition and grasp,we need to simplify the expression of the scene.Therefore,an accurate and fast indoor scene segmentation is very significant for indoor mobile robot.In this paper,the regions above the horizontal plane were regarded as region of interest;then the color and depth information were used to detect the supporting area in indoor scene;finally,the regions above the supporting surface were retained to simplify the indoor scene.Firstly,the pros and cons of the measuring distance based on vision were compared,the Kinect was chosen to acquire the depth and color information.Secondly,the depth data was filtered and get the initial foreground mask.Thirdly,the filtering depth data and 2D image coordinate were used to calculate the three dimensional coordinates and normal vector of indoor scene.Then,based on the two dimensional coordinates and normal vector of indoor scene,the super pixels segmentation were accomplished.By set the conditions of super regions fusion,the support surface was detected.Finally,based on the support regions,the initial foreground mask was updated.Refer to the foreground segmentation algorithm,the final indoor scene segmentation was completed.In order to improve the time of scene segmentation algorithm,the GPU was used to accelerate the implementation of algorithm.The innovation of paper was followed:(1)considering the characteristic of mobile robot,an initial depth threshold was set to simplify the indoor scene,the depth filtering and the calculation of normal vector were accelerated to improve the real-time of the segmentation of indoor scene;(2)according to the actual conditions,the distance metric function was modified to improve and accelerate the classic SLIC algorithm.Then the super pixels fusion was used instead of the single pixel fusion method to detect the plane to increase the speed of plane detection.The indoor scene segmentation algorithm was tested in multiple scenes.The support regions were detected fast and accurately.The background was removed while the object region above the support region were reserved.Then,the time-consuming of segmentation result based on the whole scene and local scene were compared to demonstrate the real-time of algorithm.The frame rate based on the whole scene is 10 frames per second while the frame based on the local scene is 17 frames per second,it showed that the scene segmented by depth threshold can effectively improve processing speed of the algorithm.Besides,this article also compared the time of the plane detection based on the region growth of single pixel and based on the region growth of super pixels,and result showed the time-consuming was reduced from 20.7ms to 15.3ms.It verified the validity of the proposed algorithms.To validate the sense of scene segmentation that the segmentation will improve the accuracy of the key points matching,this paper compares the matching results of different objects in the original complex scene and the segmented scene.