3D Instance Segmentation On RGB-D Images And Point Cloud Images

3D images can fully describe the geometry structure of the scene.The development of deep acquisition technology has made 3D images gradually penetrate into the fields of medical treatment,industrial production and life entertainment.As a task with both object detection and semantic segmentation,instance segmentation can give the position information and category information of each object in the scene,which has broad application prospects.Instance segmentation on 3D images is a challenging visual task.Existing instance segmentation frameworks are mostly based on 2D RGB images,without considering the use of depth information and 3D geometry information.Therefore,this thesis is aiming at RGB-D image and point cloud image.We design the corresponding network framework to solve the task of instance segmentation on these two images.The main work of this thesis is as follows:(1)We expound the definition of instance segmentation and the research status of 3D image segmentation.And we analyze the difficulties of this research topic.At the same time,we introduce different depth acquisition techniques and 3D image representations,as well as feature extraction methods on different 3D images.(2)To solve the RGB-D image instance segmentation,we propose the double pyramid feature fusion network.The method constructs two pyramid deep learning networks with different complexity to extract RGB and Depth features of different resolutions,then add two features of corresponding resolution.In this way,we change the input features of RPN to complete RGB-D images instance segmentation.The experimental results show that our model can get satisfactory RGB-D instance segmentation precision.(3)To solve the point cloud image instance segmentation,we propose the octree group proposal network.The method uses octree convolution module to perform the convolution operation on points in the XYZ direction,and stack multiple convolution modules to obtain features with different scales.We propose the softmax-hinge loss function to improve the accuracy of the similarity matrix.The experimental results show that our model can get better instance segmentation precision on point cloud.