Tracking and analysis of articulated motion with an application to human motion

Articulated motion is a subset of non-rigid motion in which the object of interest is composed of several rigid components connected to each other by ball and hinge joints. The human body, many animals and insects, and machinery all exhibit such motion. This dissertation addresses the problem of vision-based tracking and analysis of this type of motion. The importance of this problem can be seen in many application domains including surveillance, traffic monitoring, entertainment, user interfaces, medicine, sports, video annotation, and image compression. This dissertation deals with two important subproblems of the general problem: whole-body tracking and motion recognition. In whole-body tracking, the body is tracked as one unit without paying attention to the details of the posture and limbs. Current solutions to this problem suffer from being too sensitive to small changes in the environment. We present a novel approach which reduces these restrictions significantly. This is achieved by separating the concepts of a blob from that of a body and by tracking each independently while maintaining a many-to-many relationship between the two. The approach makes use of the Extended Kalman Filter and outputs trajectory information in world coordinates. The method was tested by tracking pedestrians in a variety of environments and achieved real-time performance and a high degree of robustness. Motion recognition is the high level problem of classifying an action taking place in a video sequence into one of several action categories. Most of the present approaches attempt to perform three-dimensional reconstruction of the articulated shape prior to recognition, which is an inherently difficult problem made even more difficult due to the non-rigidity of the articulated object. We argue that reconstruction is not a necessary step that must precede motion recognition. We present a novel motion-based approach for motion recognition which can be generalized to recognize any articulated motion. In our approach, an action is first represented by a sequence of efficiently computed motion features which are then mapped to a manifold in eigen-space where recognition takes place. Extensive experimentation with human subjects performing different actions demonstrated the effectiveness of our approach.