Research On Dense Estimation Of Motion Vector Field And Motion Object Extraction In MPEG-2 Compressed Domain

Video object segmentation is referred as the process to extract semantically meaningful object from video sequence. It has wide potentials in applications such as intelligent video surveillance system, target recognition and object-based video ab-straction etc. The research on video segmentation originates from pixel-domain. Al-though it is of high accuracy of object boundary, the pixel-domain based technique precision is often time-consuming, which make it hard to meet the requirements of real time application. Video is usually compressed for storage and transmission. For coded video stream, if it is decoded to pixel domain so as to exploit the pixel domain object extraction technique, the computational complexity will increase significantly, which makes it even hard to achieve real time performance. Therefore, the hot topics in motion object extraction has transferred from pixel domain to compressed domain in recent years. In other words, object extraction is directly performed in compressed domain.The motion information can be obtained from the video vector of every macrob-lock by partially decode video stream. It avoids time-consuming operations such as motion estimation/compensation, which will do benefit to real-time extraction of video objects. However, the accuracy of compressed-domain based motion object ex-traction is lower than pixel-domain based approaches, since the available motion in-formation is limited and there exists errors between real object motion and obtained motion vector field. The key is dense and accurate estimation of motion vector field in compressed-domain to improve the segmentation accuracy.This thesis researches on the optical flow approximating, i.e., the dense and ac-curate estimation of optical flow field from motion vector field easily available in video stream. Main contributions are summarized as follows:First, after a brief introduction of MPEG-2 video coding standard, the optical flow estimation and block-based motion estimation are compared. Moreover, the ex-isted work about compressed-domain optical flow estimation is surveyed.Second, a scheme for dense and accurate estimation of optical flow field is pro-posed. By utilizing motion information easily available in MPEG-2 com-pressed-domain, i.e., motion vectors and DCT coefficients, especially AC coefficients, an approximating of optical flow within the MPEG-2 compressed domain is presented. Motion vectors are pre-processed and DCT coefficients of P and B frames are esti-mated to reconstruct DC+2AC image. Black's optical estimation framework is ex-ploited to estimate a preliminary optical flow field, which is refined with a confidence measure to retain motion vectors with high confidence. They correspond to the boundary map of moving objects,thus the approximation of optical flow field in MPEG-2 compressed domain is achieved. Simulation results demonstrate that the proposed approach can solve the aperture problem, which is quite common in optical flow estimation, to some extent and provides a good analysis tool for motion object extraction.Third, a compressed-domain object extraction algorithm based on optical flow approximation is proposed. Global motion estimation and iterative rejection are fur-ther utilized to separate foreground and background motion vectors. Region growing with automatic seed selection is performed to extract the roughly accurate object boundary by motion consistency model. Partial decoding the boundary blocks is used to refine the object accuracy. Experimental results on typical test sequences demon-strate that the proposed approach can achieve pixel-level accuracy for com-pressed-domain video object extraction.