Research On Object Detection, Tracking And Behavior Recognition In Video Sequences

As the main research tasks in video surveillance, object detection, tracking and behavior recognition are widely studied in computer vision. They not only have great practical significances, but also play important roles to other topics' progress in computer vision. The ultimate goal of object detection, tracking and behavior recognition is to give the machine vision system human's cognitive ability, so as to find, track and recognize behavior of the target in image sequences. After more than tens years' development, great progress has been made in this field especially during the past ten years. However, practical experience has shown that video surveillance technologies are currently far from mature. A great number of challenges need to be solved before one can implement a robust video surveillance system for commercial applications.This thesis tries to get insights on some key issues in video surveillance; these issues include face detection, human detection in infrared images, object tracking and human behavior classification. Some important technologies and solutions are studied which are necessary for robust and practical video surveillance systems. The main contributions of this thesis can be concluded as follows.1) A face detection algorithm is proposed based on the color and geometric information. In this method, a geometric model is designed to describe geometric relations between the face region and the hair region. After a coarse detecting for the skin likeness regions and hair likeness regions, the geometric constrains between them is adopted to detect the face and the hair. Different detecting results indicate that the proposed method is both efficient and robust.2) The pedestrian detection problem in infrared images is solved from two aspects: pedestrian detection in single image and pedestrian detection in image sequences. For pedestrian detection in single infrared image, the regions of interest (ROI) are located based on the high brightness property of the pedestrian pixels, and then histograms of oriented gradients (HOG) are adopted to describe the ROI. Taking HOG as input vector, the pedestrian region is detected through Fisher linear discriminant and Bayesian classifier. For pedestrian detection in infrared image sequences, we first adopted a GMM (Gauss Mixture Model) to construct the background model, and then on the basis of segmenting the forward object, a shape-based human representing vector was designed. Taking account of occlusions among multi-humans, the intensity projection curve was applied to separate single body from occlusions. Finally, we took human shape vectors as samples, training a SVM (Support Vector Machine) classifier to detect the humans among foreground objects. Experimental results show that both of the two proposed methods are robust and efficient.3) Two algorithms are proposed to improve the performances of Mean-Shift tracking framework. On the one hand, according to the poor tracking ability when adopts static feature model, this thesis presents an adaptive feature generating model based tracking program. In this program, the object is seemed as valid tracking signal, on the contrary, the background is seemed as noise; after constructing the likelihood maps, a local SNR (Signal Noise Ratio) is computed to evaluate the tracking ability of current feature space, and the feature space with maximal SNR is selected as the optimal tracking feature space. On the other hand, an improved Mean-Shift based tracking algorithm is proposed to solve the poor tracking ability problem in occlusions. A time-invariant system is used to describe the movement of the target during a short time sequences, and through Kalman filter we identify this system so as to make it have ability estimate the coming states while occlusions taken place. The whole tracking system can divided into two parts: a Kalman parameter identifying system based on the object tracking and a Bhattacharyya coefficient analyzing system based on the Kalman state estimating; in the tracking process those two parts run by turns according to different cases. Experiment results of variant video sequences demonstrate that the proposed methods are more robust and feasible than the classical one.4) An algorithm for fusing multiple cues adaptively in particle filter tracking framework is proposed. Though it is noted that the fusion of multiple cues will lead to an increased reliability of the tracking system, most of current tracking algorithms are based on single cue and are, therefore, often limited to a particular environment. This thesis present a novel multi-cue based tracking method under the particle filter framework. Taking account of both the practical distance and the Bhattacharyya distance between particles and target, a parameter which called Relative Discriminant Coefficient (RDC) is presented to measure the tracking ability for different features. Multi-cue fusion is carried out in a reweighing manner based on this parameter. Experimental results demonstrate the high robustness and effectiveness of our method in handling appearance changes, cluttered background, illumination changes and occlusions. 5) A periodic motion analysis based human action recognition algorithm is proposed. Recognizing human action is a critical step in many computer vision applications. In this thesis the human behavior classification problem is addressed from a periodic motion analysis viewpoint. Our approach uses human silhouettes as motion features which can be obtained very efficiently, and subsequently the periodic motions are obtained by measuring the human shape's deformation. After a periodic analysis, each action unit is represented as a closed loop in the projection space and matching is performed to deicide the action's type by computing the distances among these loops. To demonstrate the effectiveness of this approach, human behavior classification experiments were performed on an open dataset. Classification results are very accurate and show that this approach is promising and efficient.Above proposed novel ideas in this thesis are try to solve three basic problems in video surveillance research: object detection, object tracking and object behavior recognition, and provide theoretic basis for commercial applications. These proposed novel ideas are associated with each other according to the research level in video surveillance.