An Automatic Video Multiple Object Segmentation Based On Higher Order CRF

Video segmentation is classified as fully supervised, semi-supervised and unsupervised based on different initialized methods. The fully supervised and semi-supervised segmentation take the user labeling pixels as training samples to learn segmentation model. Though the model enjoys good fitness, fast speed and relatively accurate segmentation, it requires pixel-level labeling which is very time-consuming and labor-intensive. And ever more mutually labeling is needed to adjust segmentation result, which increases user’s burden. Besides, it is sensitive to user labeling to train foreground and background model. Therefore, the traditional interact approaches only fit to video editing, not adapt to applications based on automatic segmentation. Unsupervised segmentation is very difficult to achieve automatic segmentation without human prior seeds because it fully depends on the performance and reliability of algorithm. Since the foreground and background binary segmentation result cannot provide enough semantic information for scene recognition and understanding, so we refer to multiple segmentation to obtain object-level result.We study the referred problems of automatic video multiple object segmentation, and make some contributions as follows:(1) The proposed multiple segmentation method is extended from common binary segmentation. We utilize initialization as seeds to train a GMM for each class based on RGB color. We also construct a strong multi-class classifier on SIFT and texture features. The dual models are utilized to estimate the probability, which makes our algorithm robust.(2) We proposed a method for automatic initializing. The common video segmentation utilizes ground truth as initialization or needs user to label initial seeds, which adds users’ burden. And we combine heat diffusion based segmentation and salient region detection to segment the first frame, which is unsupervised. Then the rough result is taken as training seeds.(3) Time is an important cue for video segmentation, so we introduce supervoxel based higher order term in CRF since supervoxel contains space-time cue. The so-called supervoxel are composed of consistency labeling in spatio-temporal neighborhood. So accurate object boundary is obtained.