Study On Motion Estimation And Moving Object Segmentation In Object Based Video Applications

With the development of the information and internet technologies, information data are currently increased rapidly. It is an important research topic how to manage and utilize video data. Furthermore, now users have additional needs of object-to-object interactions. Among these video applications, object-based video applications are improved significantly, such as object-based coding and video retrieval. In these applications, video object segmentation is a key problem. It is a key step from video processing to video analysis. Because of the importance of video object segmentation in object-based applications, this thesis investigates this problem. In object segmentation, motion features are utilized to improve segmentation accuracy. Furthermore, motion features are usually used as features in object-based application. Motion features represent temporal correlation of video. So this thesis also investigates the problem of motion estimation.First, this thesis reviews video object segmentation algorithms and further analyse spatio-temporal segmentation algorithms among these algorithms. Optical flow estimation is considered as an important kind of the motion estimation algorithm. This thesis reviews optical flow algorithms and do further research on the optical flow theory and its models. Some main problems in optical flow estimation are discussed.Since global optical flow algorithms utilize global smoothness which leads to the filling-in effect, they can produce dense flow fields. Among the global methods, the pioneering algorithm-HS algorithm is probably the most popular algorithm because of its simplicity and reasonable performance. However, it has the limitations of illumination sensitivity, unreliability of local averages, and blurred motion boundaries. We extend HS algorithm in the third chapter. Since illumination varies slightly in a short time in video sequences, a simple entity-based illimination prefiltering method is proposed. This method slightly adjusts the brightness of each pixel to make the model satisfy the illumination invariation condition. A confidence measure to assess reliability of optical flow is defined according to the bidirectional symmetry of forward and backward optical flow. Thus, a confidence based optical flow algorithm is proposed. In this algorithm, flow estimates with higher reliability have greater contributions than those with lower reliability to the averages. It leads to reliable and antisotropic filling-in effect. The algorithm keeps the simplicity of the iteration formula. Since both image-driven and flow-driven methods have complementary advantages and limitations, a region-based method combining image-driven and flow-driven methods is proposed to preserve motion boundaries. Further, we adopt this confidence as a measure to assess the reliability of optical flow fields and extend the energy-based confidence measure to non-energy based optical flow.Usually, the traditional OFE is only applied in estimation of slow-speed motion. However, the conventional OFE has a great approximation error in high-speed motion. We analyse the liminations of the traditional OFE in the fourth chapter and propose compensated OFE based two steps algorithm to estimate high-speed motion. By predicting optical flow, the model is expanded by Tayler series expansion around predication locations near true locations. It can reduce the approximation error. Furthermore, we use the second order item to compensate the OFE in order to further reduce the error. We analyse the non-quadratic smoothness method. Since the optical flow algorithm that directly utilizes the non-quadratic method has great complexity and heavy computational cost, we apply the non-quadratic method in computation of local averages. Thus, the proposed discontinuity preserving method can ulilize the non-quadratic method while keeping the simplicity of its implementation.Spatio-temproal segmentation is an effective segmentation method. First, we analyse this kind of the segmentation method in the fifth chapter. Then, an efficient spatio-temporal segmentation scheme to extract moving objects from video sequences is proposed. For localization of moving objects, a block-based motion detection method considering a novel feature measure is proposed to detect changed regions. These changed regions are coarse and need accurate spatial compensation. An edge-based morphological dilation method is presented to achieve the anisotropic expansion of the changed regions. Furthermore, to solve the temporarily stopping problem of moving objects, the inertia information of moving objects is considered in the temporal segmentation. The spatial segmentation based on the watershed algorithm considers the global information to improve the accuracy of the boundaries. To reduce over-segmentation in the watershed segmentation, a novel mean filter is proposed to suppress some minima. A fusion of the spatial and temporal segmentation results produces complete moving objects faithfully. Compared with the reference algorithms, the fusion threshold in our scheme may be fixed for different sequences.To reduce the complexity of segmentation, we further investigate spatio-temporal segmentation and propose a spatio-temporal compensation based segmentation algorithm in the sixth chapter. The temporal segmentation localizes moving objects by comparing the motion vector of each block in each frame with the corresponding global motion vector. To estimate the global motion vector accurately, an outlier rejection (OR) based method is presented. Furthermore, the temporal compensation utilizing the temporal coherence of moving objects is considered in the temporal segmentation in order to solve the temporarily stopping problem. The detected moving object regions usually have discontinuous boundaries and some holes. The region growing algorithm with a distance constraint is utilized to compensate the coarse object regions in the spatial domain. By using a fusion module, moving objects are extracted. Since moving object detection in video surveillance is an important step, we propose an automatic object detection algorithm based on spatio-temporal compensation for video surveillance. This detection algorithm combines some methods proposed above. The proposed detection algorithm can extract moving objects as completely as possible and its computational cost is acceptable for surveillance systems.