Video Object Segmentation Algorithm Based On Conditional Random Field Models

Video object segmentation is a challenge for computer vision, which is a critical step in many applications such as video surveillance, human-computer interaction, as well as video editing. Efficient and accurate video object segmentation algorithm can greatly reduce the difficulties of subsequent applications. Video object segmentation algorithms can be divided into automatic video object segmentation algorithms and interactive video object segmentation algorithms. The automatic video object segmentation algorithms are more useful, and the video object segmentation algorithms proposed in this dissertation are all automatic video object segmentation algorithm. The automatic video object segmentation algorithms can be separated into segmentation algorithms at low level, segmentation algorithms at intermediate level and segmentation algorithms at high level according to processing levels for video sequences. At these three levels, we proposed corresponding segmentation algorithms. The first chapter in this dissertation is the research background, and a modeling method based on weighted models for shadows and foreground is presented in Chapter 2. A video threshold method based on 2D conditional random field model is proposed in Chapter 3, and in Chapter 4, a video object segmentation algorithm based on hierarchical conditional random field model is presented. A multiple video object segmentation algorithm with incorporating object recognition information is proposed in Chapter 5. In Chapter 6, a system of traffic flow analysis based on video object segmentation algorithm is presented.Contributions of the dissertation are listed as follows:1) A modeling method based on weighted models for shadows and foreground is presentedSegmentation algorithms at low level usually model background, shadows and foreground of video sequence at pixel level. Active shadows are the factors affecting video segmentation quality, efficient shadow removal method can improve the segmentation quality. A modeling method based on weighted models for shadows and foreground is presented in Chapter 2, which can detect active shadows indoors and outdoors. The shadow models model the active shadows of video sequence at pixel level and compute their probabilities. The foreground models compute the probability distributions of foreground by the weighted method. The probabilities of these models are capable of providing essential data for segmentation algorithms at higher levels in the following chapters.2) A video threshold method based on 2D conditional random field model is proposedThere are usually many false classifications in the results of segmentation algorithms at low level. These false classifications can be corrected by segmentation algorithms at intermediate level that incorporate neighboring relationships of video sequence. A video threshold method based on 2D conditional random field model, which is a segmentation algorithm at intermediate level, is proposed in Chapter 3. The proposed algorithm defines feature functions for neighboring relationships of video sequence and for background, shadow and foreground models described in Chapter 2, it constructs a 2D conditional random field model by which these feature functions are modeled. The inference algorithm is adopted to solve the models for obtaining the final segmentation results.3) A video object segmentation algorithm based on hierarchical conditional random field model is presentedVideo threshold method based on 2D conditional random field model can eliminate false classifications produced by segmentation algorithms at pixel level. But if pixel blocks of false classification are too large, the 2D conditional random field model can not correct these false classifications. To solve this problem, we introduce local neighboring relationships and global neighboring relationships for video sequence in Chapter 4, and extend hidden conditional random field model to construct a hierarchical conditional random field model by which local and global neighboring relationships are modeled.4) A multiple video object segmentation algorithm with incorporating object recognition information is proposed Segmentation algorithms based on background modeling for video surveillance system have some limits, which need assume background are relative static in video sequence, and need deal with active shadows, and is difficult to segment multiple objects. A multiple video object segmentation algorithm with incorporating object recognition information, which is a high-level segmentation algorithm, is proposed in Chapter 5. The proposed algorithm can segment the objects occluded each other and the partial objects. The algorithm consists of training section and segmentation section. At the training section, the algorithm constructs feature dictionary and learning parameters of hierarchical conditional random field models from training data. At the segmentation section, the algorithm models feature functions via hierarchical conditional random field models to obtain segmentation results which combine the top-down information and bottom-up information.5) Research on video based traffic flow analysis systemVideo based traffic flow analysis methods which have many advantages over traditional methods are proposed in recent years. A system of traffic flow analysis based on video object segmentation algorithms is presented in Chapter 6, the system mainly consists of object segmentation step and object tracking step. The system selects segmentation algorithms according to different scenes at the object segmentation step. At the object tracking step, to tracking the objects, the system searches the shortest frame distances among objects at neighboring frames which are provided by the object segmentation step, and color features are adopted to assist for tracking. We implement a demo of video based traffic flow analysis system that is capable of measuring vehicles and providing traffic flow levels.