Moving Object Segmentation From Video Sequences

Moving object segmentation from video sequences is a key step in many computer vision applications, such as video surveillance and behavior understanding. The goal of moving object segmentation is to extract the silhouette of the object of interest from static or dynamic background. Many segmentation algorithms have been presented in recent years. Earlier algorithms are always fallen into the category of parametric modeling. The latest algorithms are all based on the MAP-MRF segmentation framework. Experiments show segmentation noise can be successfully removed by the MAP-MRF framework compared with those parametric modeling technique.Besides of segmentation noise, there is another serious problem of segmentation error: splits and defects. Splits and defects are caused by the color similarity between the foreground and background. When the foreground shows similar color to the background, many foreground pixels are misclassified as background both by the parametric modeling and MAP-MRF framework. The color similarity problem is the topic of this paper.We deeply discussed the color similarity problem under the MAP-MRF framework. The analysis reveals that the essence of the color similarity is the existence of a confusion point and the inaccuracy of foreground modeling. Accordingly, we presented the solutions for the color similarity problem: shifting the confusion point, and improving model accuracy.The work in this paper is all based on the solutions of the color similarity problem. The work in this papar is as follows:1. We developed weighting methods to deal with the confusion point. The weighting methods are designed to shift the confusion point towards the right side. The color similarity energy term is specially designed to detect foreground pixels showing similar color to the background.2. We developed new models to deal with the model accuracy. Some new models take advantage of the motion of the moving object to compensate the model inaccuracy due to the motion of the moving object. Some new models take advantage of the shape of the moving object to compensate the model inaccuracy due to the deformation of the moving object.3. We extended foreground segmentation from single object to multiple objects by the use of the pedestrian detector and tracking algorithm.4. We presented new metrics for model accuracy evaluation.