Research On Video Segmentation And Feature Extraction In The H.264 Compressed Domain

Video segmentation and feature extraction technologies are very important and challenging issues in the video analysis and retrieval field. At present, most of the videos are stored or transmitted in compressed domain. H.264/AVC (H.264 for short) is a new generation video standard set up by ITU and ISO/IEC. It is broadly adopted by video compression and used in storage and transmission applications. For traditional content-based video analysis and retrieval applications, the video segmentation and feature extraction algorithm are all based on pixel domain. As the increase of video quantity and the improvement of resolution, the video analysis based on pixel domain is restricted by its calculation resources. Therefore, some researchers have proposed video segmentation and feature extraction algorithm based on compressed domain, and most of these algorithms are carried out in the MPEG compressed domain. However, these algorithms can not fully fit the situation of H.264 due to many new encoding tools introduced by H.264 coding standard. Therefore, it is of the theoretical significance and practical value to carry on in-depth study as how to use the information extracted from the H.264 video bit-stream in making accurate video segmentation and efficient feature extraction. Based on this background, the video segmentation and feature extraction algorithm based on H.264 compressed domain are addressed by this paper.In Chapter 1, the significance of the research work is presented together with a brief summary of the present research status. Then, the main research content and the structure of the thesis is introduced.Chapter 2 proposes an adaptive threshold scene change detection algorithm based on the bit count information of the H.264 macro-block. The algorithm consists of the scene cut detection and scene gradual change detection. The bit counts of intra macro-blocks, the bit counts of inter macro-blocks and the ratio of skip mode macro-blocks is used to detect the scene change situation with an adaptive threshold.Chapter 3 is the most important section in this thesis. The spatial object segmentation algorithm in the H.264 compressed domain is investigated. A Block Residual Energy Image(BREI) based modeling the background and segmenting foreground object in H.264 compressed domain is introduced. The information extracted from I frames are used to build the BREI. Then, the BREI is filtered using the directional features of the intra prediction information. Finally, the mixture Gaussian model is used to model the background of the BREI and segment the foreground objects. For the moving object segmentation, the motion vectors are the most important information in the algorithm.In order to improve the robustness of the motion vectors, they are filtered according to the extra information from intra modes, intra prediction residual energy, and inter partition modes. The Gibbs potential functions and Cellular Neural Network based methods are applied respectively on these filtered motion vector fields, and the moving objects from the compressed bit-streams are extracted.Chapter 4 proposes an algorithm of extracting block edge pattern by intra prediction mode from H.264 I-frame bit-streams. Five block edge patterns are extracted based on intra prediction mode and prediction coefficients, and are accorded with the edge descriptor in MPEG-7. The feature is suitable for content based image retrieval algorithms.Chapter 5 proposes a video object tracking and trajectory feature extraction algorithm in the H.264 compressed domain. Based on the object segmentation results from chapter 3, the objects'initial position is selected. The tracking process is carried out afterwards based on the confidence region and the consistency of the moving direction. The trajectory feature of the moving object is generated by the tracking results and divided into direction feature and detail feature. The Hausdorff distance is used to measure the similarity of trajectories. Its effectiveness is proved through the experiments.In Chapter 6, according to the large number of compressed video data in video surveillance system, the algorithms of scene change detection, background modeling and foreground object segmentation, moving object segmentation, edge feature extraction and motion feature extraction are combined in a video surveillance retrieval system. The prototype of the video retrieval system in H.264 compressed domain is constructed.The final chapter concludes the new achievements of the whole research and the prospect of the future research.