Depth Data Based Human Video Editing

In this paper, we present a new approach for human video tracking and editing. In-stead of traditional method based on frame by frame tracking which always use feature tracking or optical flow, we registered each frame to a template frame to get a global con-sistent trajectories for video editing.To solve the problem of image registration of large deformation, we first register 3D depth point cloud captured by Kinect together to get an initial correspondence and then use image information to optimize the registration.For non-rigid registration of point cloud of large deformation, we present a body based method. We first reconstruct human bodies for each frame from depth video, and then use bodies to register each frame to template frame for an initial global consistency trajecto-ries. At last, a large optical flow method and optimization method applied to get the final trajectories. Then we can edit template frame to edit all frames easily.There are several contributions of our new framework. First of all, we can reconstruct bodies sequence from Kinect data fast and automatically based on a human space; Second, we present a body based registration method which can register two frames of large non-rigid deformation without precise template. Last but not least, we first propose a depth data based video editing framework, our framework gives a global consistent trajectory. And we can get good results for kind of bad cases that failed in traditional method, like simple texture, non-rigid motion and occlusion and so on.