Study On Object Segmentation In Stereo Images And Stereo Videos

Stereo systems are designed to emulate human stereo perception, which have many applications. One major obstacle for the application of stereo videos is the extremely large amount of data associated with a stereo sequence. To enable the storage or transmission of stereo sequence at a reasonable cost, substantial compression of the data must be accomplished. There are mainly two methods to compress stereo video, block-based coding and object-based coding. Block-based coding has the merits of simplicity and robustness, but may cause blocking artifacts. Object-based coding is very desirable as it can avoid blocking artifacts, and has higher coding efficiency. Furthermore, it can describe a scene in a structural way. Therefore, it has been an active area of research as the developing trend of stereo video compression scheme. As the name implies, object-based coding is a concept of video compression that relies on the detection of foreground objects in a video scene. Therefore, video object segmentation is a crucial step in object-based stereo video coding.Object segmentation is also an important step in many computer vision and multimedia tasks. In the study of images, images are split into several regions. The certain regions which draw more attention of researchers are called objects or foreground, and the rest regions are called background. Image segmentation is one of the most difficult tasks in image processing, for images are influenced by illumination change, background confusions and occlusions, etc. There isn't a universal segmentation algorithm for all kinds of images. Therefore, object segmentation in stereo is a problem full of challenge.Object segmentation algorithms are derived from image segmentation and one-channel video segmentation algorithms. Some researchers segment objects in one channel using traditional image segmentation algorithms, then get the objects of the other channel based on stereo matching. Some other researchers do the segmentation procedure on the depth map. This method tends to be more accurate, for the depth information is quite close to the true object boundary.This paper mainly works on the object segmentation problem in stereo images and videos. For stereo images, an object segmentation algorithm based on disparity map is proposed. A stereo video segmentation algorithm combining disparity map segmentation and twice frame difference segmentation is also proposed in this paper. Both of them have two steps, described as follows:(1) The first segmentation procedure is based on the disparity map, for disparity map contains the depth information associated with the 3-D scene. One can get disparity map after stereo matching. For the parallel camera configuration, the epipolar lines are parallel to horizontal scan lines, so one can constrain the search within the horizontal scan lines. For the converging camera configuration, the epipolar lines are parallel to horizontal scan lines after epipolar line rectification. Therefore, a one dimension window is employed to do the match procedure. In order to reduce the computation, the stereo matching is based on the undersampled stereo images. Based on the disparity smooth constraint, objects in different disparity planes are located. After morphological processing and upsampling, we can get the first segmentation result.(2) The second segmentation procedure of stereo images is based on the first segmentation result. We take advantage of the characteristics of boundary pixels: first, their matching error is much bigger; second, the disparity on the left hand side is different from disparity on the right hand side. Then a one dimensional window is employed to search for matches in corresponding scan lines. The searching bound is constrained by the disparity of background and object. If the matching error is smaller than a giving threshold, the pixels are matching points, or the pixels belong to the object boundary. Combing the second segmentation with edge detection, we get the final object. This algorithm is efficient for stereo images with small occlusions.(3) Moving object segmentation in stereo video performs in the regions derived from the first segmentation procedure, based on the twice frame difference algorithm. By subtracting present frame with former frame and latter frame, we get two frame difference images. Higher-order statistic is used to judge whether each pixel in frame difference image belongs to moving object. Then moving object in the two frame difference image is extracted. We take the intersection of the moving object in these two images, and this intersection is the moving object of the present frame. Combing moving object segmentation with edge detection, we get the final moving object. Using this algorithm, we can segment moving objects in different disparity planes. Over segmentation can be avoided in the case that there is motion interaction between moving objects or motion information are involved in the background.The results show that algorithms proposed in this paper perform well in object segmentation in stereo images and stereo videos. Meanwhile the algorithms are fast, and easy to implement.The second segmentation of stereo images is efficient for stereo images with small occlusions. When there are large occlusions, complicated occlusions detection and disparity compensation must be taken. Our work is based on gray information of images and videos. In future work we anticipate that by involving color information, it should be possible to get more accurate segmentation results.