Object Segmentation And Tracking In H.264 Compressed Domain

Video object segmentation and tracking is an important part of many computer vision applications,such as motion-based recognition,human-computer interaction,automatic surveillance,and traffic monitoring.Segmentation is that separating the foreground region from the background in the image frame.Tracking can be defined as the problem of estimating the trajectory of an object as it moves around the scene.Segmentation and tracking algorithms can be divided into two categories based on their operational domain: pixel domain algorithm and compression domain algorithm.The pixel domain algorithm is characterized by high accuracy but high computational complexity.The higher computational complexity limits its application in scenarios where several video streams need to be processed in parallel.Compressed domain algorithms use data encoded in a compressed video bitstream,such as motion vectors,block coding modes,or transform coefficients of motion compensated prediction residuals.The compressed domain algorithm avoids full video decoding and usually has a lower computational cost.This article separately introduces the video object segmentation method and video object tracking algorithm in the compressed domain.The main work is as follows:1.A block refinement method based on block coding mode is designed.The basic framework of the algorithm is to aggregate blocks with non-zero motion vectors into the connected foreground areas and use the target area tracking to remove noise areas in the foreground area.By using the block coding mode to further refine the boundary area,the segmentation accuracy of the algorithm is improved.2.The I-frame information in the compressed domain is used to track the target based on the tracking algorithm of the space-time Markov model.This algorithm only uses the MV and block coding modes,and utilizes spatial and temporal correlation of the moving target to perform frame-by-frame tracking.After the completion of each image group,the label field is updated using the compressed domain I-frame information.This algorithm improves the problem of degraded tracking accuracy due to error accumulation.3.Based on the different characteristics of the H.264 block scale,the experiment verifies the different tracking results of the target tracking method based on the space-time Markov model under different block sizes.The results show that the smaller the block,the higher the precision.For different targets with different motion modes,the tracking results are also different.It lays the foundation for future dynamic block tracking.