Object and Action Recognition Using Poisson Based Shape Representations

This thesis concerns with the computer vision tasks of shape representation, visual object detection, categorization and segmentation as well as human action recognition. Specifically, we have developed methods that extract and represent explicit shape information from in images and video sequences and incorporate this information in the process of recognition. The shape of an object is an important cue for recognition and is especially informative in cases where object appearance varies significantly due to variations in color, textures and changes in lighting and viewing conditions. This work employs a Poisson-based shape representation to characterize shapes of 2D silhouettes in the context of object and action recognition and extends this representation in several ways.;First, we address the problem of object recognition in cluttered scenes. Unlike many common methods, which are based primarily on appearance, we introduce a shape-based detection and top-down figure-ground delineation algorithm. Our method utilizes dense regional Poisson-based shape descriptions of image segments, emerging as a result of data-driven, hierarchical image segmentation. We further account for the partial silhouettes (shapes) and the incomplete boundaries frequently produced by image segmentation processes. We employ probabilistic shape modeling and use statistical tests to evaluate ensembles of partial shape hypotheses to identify the presence of objects of interest in the image and to delineate foreground objects from their background.;Second, we generalize the Poisson-based 2D shape representation to describe actions video sequences for the task of action recognition. Our method is based on the observation that 2D silhouettes of moving humans concatenated in time induce a 3D shape in the space-time volume that captures both the spatial information about the pose of the human figure at any time, as well as its dynamic motion information.;Next, we extend the Poisson-based shape representation approach to handle gray scale images without the need for prior segmentation. Our approach incorporates uncertainty in the identity and the correct location of object contours and utilizes the entire distribution of random walk hitting time, encoding richer information about the shape.;Finally, as a side research topic, we examine the use of Poisson-based, implicit shape representations in medical imaging applications. We use these representations for registration and segmentation and apply them to liver and caudate nuclei CT images.