Online Video Segmentation

In recent years, along with the development of hardware, the applications of video tech-nique have extended to many new areas, most of which are real-time systems, including augmented reality, teleconferencing, etc. Some of these systems need to know the accu-rate segmentation of the interested object(s) in order to be able to process correspond-ing regions in a special way. Compared with offline video segmentation, online video segmentation not only need to reach real-time speed, but also cannot involve user inter-action at online phase. Therefore, the segmentation algorithm should be very fast, and at the same time, very robust, which is very hard to be achieved in real environments.This dissertation studies a series of key problems and techniques related with on-line video segmentation, and includes the following contents:(1) A brief survey and discussion of previous works, based on which the four prob-lems are proposed, i.e. robustness in the scene of stationary background, real-time seg-mentation in dynamic scenes, automatic initialization and real-time post processing.(2) Presents a confidence-based color modeling method, which can greatly improve the robustness of segmentation methods in the scene of stationary background. By eval-uating the confidence of the global color models and the background model, the optimal combination can be achieved for individual pixels, in this way the errors introduced by ambiguous colors can be greatly reduced.(3) Presents a real-time transductive video segmentation method, which can be used for the cases of non-stationary background. The key is a novel local color mod-eling method combined with the temporal continuity, which is not only more accurate than the global color models but also can be constructed in real-time. By using this tech-nique the segmentation result can be propagated accurately without using background information.(4) Presents an automatic initialization method for online video segmentation. Tra-ditional initialization methods require either the background image or the segmentation result of the first frame, or need to be pre-trained in specific scene, which are very in-convenient in practice. The proposed initialization approach is based on a novel motion segmentation method, which does not need to be pre-trained, and can extract the fore-ground object from two adjacent frames when it is moving. (5) Presents a real-time post-processing algorithm, which can efficiently remove mi-nor errors around the boundary of binary segmentation. In order to suppress flicking, traditional methods usually simply smooth the boundary by feathering, which can not result in good effect in most cases. On the other hand, significant methods are hard to reach real-time speed. The proposed post-processing algorithm can meet the require-ment of accuracy and efficiency at the same time, and thus solves this problem very well.(6) Concludes the works of this dissertation, and presents some problems that can be studied further.