Research On Image Segmentation Combined With Depth Information

One of the long-term goals of computer vision is to be able to understand the world through visual images. With the popularity of mobile intelligent terminal equipments and advance in networks, large-scale visual data become available to ordinary users. How to analyze these visual data and then how to effectively organize, represent, manage and retrieve these visual data become a challenging task in both research and industry. To achieve this goal, image segmentation which is considered to be an effective way to achieve image understanding has attracted more and more attention.In its early days, image segmentation technique is completed based on the image appearance feature clustering. An image is segmented to homogeneous regions in appearance. However, in the real world, objects are typically defined, not by homogeneity in appearance, but by physical connectedness.It is hard to get semantic consistency object segmentation for image segmentation techniques based on appearance clustering. This has prompted people to look for new image segmentation techniques. Traditional image segmentation techniques analyzed the image as a2D pattern rather than treated it as a projection from3D world. We believe it is necessary to use depth information to guide image segmentation. The using of depth information will provide a possible solution for image segmentation.In this thesis, we consider how to perform image segmentation combining depth cue and color cue. Several image segmentation methods combining depth information and color information are proposed. The proposed methods can get semantic consistency segmentation and overcome over-segmentation or under-segmentation in traditional2D image segmentation methods based on appearance features. The main contributions are illustrated as follows:1. Appearance based image segmentation methods ignore the physical connectedness which makes it hard to avoid over-segmentation and under-segmentation for them. We propose a novel image segmentation method in which seed regions are selected based on depth discontinuities. First, we combine depth and color cues to extract depth discontinuity boundaries which correspond to object boundaries. Then over segmentation of color image is performed and color segments neighbor to the depth discontinuity boundaries in each side are selected as seed regions for different objects. Graph cut is used to optimize the energy for labeling the remaining segments which are not selected as seed regions at last. Compared to conventional methods, our approach can effectively separate different objects with discontinuous boundaries. At the same time, our method can also separate two objects contact each other (i.e. support and supported relationship). Semantic image segmentation is solved successfully.2. Depth discontinuity which corresponds to occlusion in3D world is widly used in image segmentation. Most existed image segmentation methods based on depth map segmented a scene to homogeneous regions in depth. These methods ignored the depth continuous at the connections between support and supported objects and these methods couldn’t separate support and supported objects apart. In this thesis, we proposed a support analysis based image segmentation method. We use depth map to analyze the support relation between different objects in a scene. In color over-segmentation, segments with surface normals approximately orthogonal to horizontal plane are set as support regions, the remaining segments are set as supported regions. Then we perform depth segmentation in support and supported regions, separately. Our method can separate support and supported objects apart successfully.3. Geometric structure has been widely studied as useful information in scene understanding and object recognition. Geometric classes describe the3D orientation of an image region with respect to the camera. Traditional methods only used appearance features to estimate geometric structure (currently horizontal, vertical, or sky). These methods ignored the correlation between geometric classes and depth features and made inconsistent geometric structure estimation. We proposed geometric structure estimation combining depth and appearance features. Our experiments demonstrate the incorporation of depth features allows us to achieve state-of-the-art classification results than previous works. In addition, we also improve the image segmentation using estimated geometric structure.Image segmentation is closely related with many different domains, such as computer vision, machine learning and cognitive science. We hope that our work can be helpful for relating researches.