Tracking multiple objects using multiple hypothesis tracking framework

The goal of this thesis is to develop an efficient and robust tracking framework that employs multi-level features to track multiple objects in video sequences using Multiple Hypothesis Tracking (MHT) algorithm. The framework is capable of operating with low level as well as higher level features including point-based, edge-based and model-based features extracted from images employing motion, color, shape and intensity information or known 3D models of objects.; A framework for tracking objects represented by point features using MHT algorithm is first described. A path coherence function is introduced such that it includes the direction and speed information as an additional constraint to the traditional MHT algorithm in the hypothesis generation and elimination process. Inclusion of this constraint reduces the number of hypothesis which inturn improves the computational efficiency and robustness of the algorithm. It also provides more enhanced trajectory structure in case of occlusion and close interaction of the multiple objects. The experimental results are analyzed using an evaluation and quantification protocol. This framework is used for tracking point representation of body parts (i.e., face and hands) of multiple people in the context of human computer interaction and human motion analysis in video sequences.; Representing objects by a point is not sufficient for some tracking applications since the shape and the viewpoint of the objects cannot be recovered. In addition, using more complicated features provides more information about the object and it increases the robustness of tracking. Thus, an edge-based feature tracking method using MHT in four-dimensional space is introduced and a framework for localizing and tracking multiple complex objects in video sequences is described. The framework utilizes MHT algorithm to track multiple line segments that represent objects of interest in the scene. These individual segments are matched and combined into object level using Hausdorff image matching algorithm based on given two-dimensional object models. The experiments are performed on real image sequences and the analysis of tracking results are presented.; In some cases 3D models of object are available. 3D models provide more information about the object's pose (i.e., rotation transformation) which cannot be recovered using edge information. Using 3D models as a known priori constrains the feature trackers to operate in a consistent manner. A model-based object tracking framework is also presented in this thesis. The framework employs known geometric, dynamic and appearance models of objects during tracking. The tracking problem becomes the problem of determining the pose parameters of the models given an observed image sequence. The framework iteratively updates the model parameters (e.g., position in the image plane and orientation angles) each time a new image in the sequence is observed. The operation of the framework first start with estimation of object configuration using computer graphics algorithms (i.e., hidden line removal) to determine the visual feature templates. These feature templates are then used by Hausdorff image matching algorithm to obtain the location of image features by minimizing the Hausdorff distance between model points and image feature points. Finally, these observed feature locations are then used in MHT algorithm which updates estimate of the object configuration.