Point Trajectory Estimation And It’s Application In Video Segmentation

Point trajectories are generated through sampling points in video and tracking to next frame with optical flow until being occluded, which have the ability to express long term motion information of video and address the situation that object without continuous movement. The clustered point trajectories as intermediate results are adopted to achieve action recognition, 3D reconstruction or video segmentation. This paper puts forward super point trajectory. An affinity measure method which considers the motion and texture of point trajectory is designed to obtain super point trajectory through optimization by random walk. A video object segmentation approach via dense point trajectory is proposed in this paper. And a new affinity measure method considering global as well as local information of point trajectories is adopted for trajectory clustering on basis of predecessors. Then a video segmentation framework with global motion information of video and a new method for determining background and foreground of video are proposed based on trajectories. The main work and contributions of this paper are as follows.A super point trajectory method is proposed. Super point trajectory is the set of trajectories which have same attributes and closely packed together. Super point trajectory express long term motion information of video and regard the point trajectories in same region, such as the wheel of car and the arms of people. The super point trajectory can also reflect the texture as well as contour of video. Therefore, an affinity measure method considering motion and texture information of trajectory is put forward, which is calculated by counting the times of pair trajectories go through the same superpixel as well as computing texture information of point trajectory. Then random walk is adopted to obtain super point trajectory. Next, an energy function is designed to group point trajectories with super point trajectory. Finally, the performance of the super point trajectory method is analyzed through experiments.A video object segmentation method via dense point trajectory is designed. The information of common frames of pair trajectories is regarded as local information and the average motion intensity as global information are adopted for affinity measure method. Then, landmark subspace spectral cluster method is adopted to achieve trajectory cluster results. A video segmentation framework considering global motion of video with point trajectories is created. The point trajectories been clustered not only show object regions, but also reflect the spatial and temporal continuity of video. Appearance model of video is computed by the color cues of each kind of trajectories. Next, video partition results are calculated through optimizing our energy function with graph cut. Background trajectories and foreground trajectories are indicated with clustered trajectories and used to identify background and foreground of video with partition results.