Persistent object tracking in unconstrained environments

Visual tracking has been an active and fruitful topic in computer vision these days since it is a very important component for a wide range of practical systems such as surveillance, robotics, multimedia information retrieval, human-computer interaction. In this dissertation, our goal is to track an object of interest persistently with respect to the appearance changes caused by illumination variations, viewpoint changes, and out-of-plane rotation. An object also may leave the field of view then reappear. In order to track and reacquire an unknown object with limited labeling data, we propose to learn these changes online and build a model that encodes all appearance variations while tracking. To address visual object tracking as a semi-supervised problem, we propose a co-trained cascade particle filter framework to label incoming data continuously and online update hybrid models generatively and discriminatively. Each of the layers in the cascade contains one or more either generative or discriminative appearance models. The cascade manner of organizing the particle filter enables the efficient evaluation of multiple appearance models with different computational costs, thus, improve the speed of the tracker. This proposed framework provides not only the ability to adapt to object appearance changes but also an object-specific detection capability which allows to reacquire an object after total occlusion.;One of the vital issues of visual tracking is drifting where partial occlusion is a critical factor. We propose to use the co-training framework of generative and discriminative models to detect where the occlusion occurs. Precise occlusion segmentation is then performed using Meanshift. After that, the occluded parts are recovered using the learned information from the models. We also propose to use local feature movement voting scheme to serve as a referee when there is a strong disagreement between the two models. Finally, each of the models will be updated using the new non-occluded information.;Another major factor causing tracking failure is the emergence of regions having similar appearance as the target. It is even more challenging when the target leaves the field of view (FoV) leading the tracker to follow another similar object, and not reacquire the right target when it reappears. We propose a method to address this problem by exploiting the context on-the-fly in two terms: Distracters and Supporters. Both of them are automatically explored using a sequential randomized forest, an online template based appearance model, and local features. Distracters are regions, which have similar appearance as the target and consistently co-occur with a high confidence score. The tracker must keep tracking these distracters to avoid drifting. Supporters, on the other hand, are local key-points around the target with consistent co-occurrence and motion correlation in a short time. They play an important role in verifying the genuine target.;To demonstrate the feasibility of our proposed framework, we apply it into a practice surveillance application. We use a commercial network pan-tilt-zoom camera to build a real-time active tracking system with two scenarios: (1) an object of interest is tagged, and (2) an automatic surveillance system which can detect a pedestrian, zoom to the face, detect it, and then track it to extract a high resolution face sequence for forensic purposes. Our system keeps following the object by automatically adjusting PTZ parameters to maintain the object at the center of the screen with a proper size.;Extensive experiments are shown to demonstrate the efficiency and robustness of our proposed framework. Comparisons with state-of-the-art methods are also provided. In the application of our thesis, we demonstrated our surveillance system in various challenging real-life scenarios.