Research On Video Object Segmentation And Tracking

Extracting foreground moving objects from video sequences is an important task and also a hot topic in computer vision and image processing. Segmentation results can be used in many object-based video applications such as object-based video coding, content-based video retrieval, intelligent video surveillance, video-based human-computer interaction, etc. Researching on video object segmentation technology is of great research importance and applicatory significance.Similar to image segmentation, video object segmentation is also a very difficult problem in computer vision. Its difficulty lies on two aspects: First, there is abundant spatial and temporal information in video streams. Because of the complexity and diversity of real video scenes, we cannot define one uniform model to cope with all video objects; Second, video object is a high-level semantic concept, which is difficult to be obtained by computing low-level vision features. It is hard to bridge the semantic gap between low-level features and high-level concepts. Most previous approaches for foreground segmentation did not consider the spatial coherence of the image and only make local decisions. Although satisfactory results are obtained in latest works, they usually pay less attention to the problem of cast shadows and the segmentation speed is not fast enough for real-time application when processing high resolution videos.The main content of this thesis is doing research on critical problems in spatio-temporal segmentation method. We propose an efficient foreground extraction framework by combining global motion detection, background modeling, shadow detection and Markov random field model. Our system is capable of extracting foreground layers from monocular video sequences with shadow removal accurately and efficiently. The main work of this paper is as follows:(1) A global motion detection method based on edge difference is proposed. To apply detection on video streams captured by PTZ cameras, we propose Edge Change Ratio to detect global motions. Our method has low computational complexity and high accuracy, which is very useful in real-time surveillance.(2) Initial foreground regions are used to reduce computation cost. We use Gaussian Mixture Models (GMM) to model the scene and obtain initial foreground regions. Segmentation algorithm is only implemented in these regions to reduce the computation cost. The processing speed is greatly improved by our method.(3) We propose an efficient video object segmentation framework by using quadrant-map and Markov random field model. Shadow detection is implemented on each foreground regions to generate quadrant-maps. Based on these quadrant-maps, Markov Random Field model is built on each region and graph cuts algorithm is used to get the optimal binary segmentation. To ensure good temporal consistency, we reuse previous segmentation results to build the current foreground model.(4) An experimental video object segmentation system is developed and various types of video sequences are used to test the proposed framework. Experimental results on various videos demonstrate the efficiency of our proposed method.