Video Object Segmentation In The H.264 Compressed Domain

MPEG and H.264 are the leading video coding standard these days. The MPEG standard first proposes the video coding based on the content. The video object segmentation in the compressed domain attracts attention in recent years while most work is focused on the pixel domain before. Since most videos are converted into the compressed format, the video object can be extracted in the compressed domain where the video need not be decompressed completely. At the same time, the computation load decrease and the necessary data storage reduce greatly because the processed data are much less in the pixel domain. In addition, the motion information extracted from the compressed video with entropy decoding are used as the motion characteristic and texture feature. Therefore, the fast object segmentation in the compressed domain can meet with the real time requirements which the traditional pixel domain methods fail to satisfy. The research on video object segmentation in the compressed domain has the academic meaning and applied meaning.H.264 is the latest video coding standard and its coding efficiency is twice as the MPEG standard. More and more applications turn to the H.264 in place of MPEG. The video object segmentation in this dissertation uses the motion information including the macro-block coding mode and motion field extracted from the H.264 compressed domain. On the other side, some difficulties exist in the object segmentation in the H.264 compressed domain because the extracted motion information can not reflect the true motion. These difficulties are: (1) The required segmentation information in the H.264 compressed domain is limited, the segmentation task in this dissertation is rather hard only based on the macro-block coding mode and motion filed. (2) The motion information in the H.264 compressed domain is not accurate, the macro-block coding mode cannot reflect the video background entirely and the motion field cannot represent the true motion. These problems make it hard to improve the segmentation quality. (3) Many current motion segmentation methods in the pixel domain are not easy to fulfill the requirements of both segmentation speed and segmentation quality in the compressed domain.Based on the research on the above problems, some trouble issues including motion information densification and correction, the background detection via macro-block coding mode, the improvement for efficiency and quality of segmentation via the spatio-temporal correlation, the improvement for segmentation effectiveness by combining region segmentation and region classification as well as the background detection by fast motion estimation as to simplify the energy function formulation. The major work and innovations in this dissertation are listed as following:(1) The effective motion preprocessing is proposed to resolve the initial sparse and noisy motion field. The spatio-temporal filtering is exploited to remove the noisy motion vectors. Then the backward estimation and forward projection are used to obtain the dense motion field.(2) The macro-block coding mode is used in the object segmentation to improve the processing speed because it is hard to increase the segmentation efficiency based on motion field only. The background of video frames indicated by H.264 macro-block coding mode can narrow the range of motion estimation and improve the processing speed. At last, theχ2 testing is used to extract the video object based on the estimated background.(3) In order to classify object region and background region more accurately, the method combining region growing and region classification is proposed in the dissertation. At first, the magnitude, the divergence and the curl are used to characterize the motion field, the modified statistical region growing is applied to divide the motion field into different regions. Then the 4-order moment is used for region classification and the final result is refined by the projection filtering. The way combining region growing and region classification can classify the object region and background region as well as improve the segmentation results.(4) To the problem that Markov random filed (MRF) is hard to extract object from the H.264 video with moving background. The segmentation method using Graph Cuts is proposed in the dissertation. At beginning, the static background is detected by the span thresholding while the moving background is detected by the motion estimation. The obtained information about background simplifies the energy function of Graph as well as improves the computation efficiency of Graph Cuts method. The proposed methods can be used to extract object from the video with static background or moving background. Moreover, the segmentation quality is better than the similar method.