Key Techniques Of Content-based Intelligent Video Surveillance And The Applications In Public Security

Intelligent video surveillance can smartly understand the video content, find the unusual events and alert. One of the most important applications is automatically monitoring the behaviors of suspects. This thesis focuses on this application and addresses the problems of motion detection, action recognition, face super-resolution, expression synthesis and 3D reconstruction based on learning.First, we discuss the motion detection and extraction based on incremental background modeling. An adaptive weight selection mechanism is put forward to automatically determine a weight for each frame according to the motion contained in this frame. The background model is updated using the reasonably weighted frames. This background model can adapt to the dynamic scene well and generate good background image even when the background scene contains complex motions.Second, to automatically understand human behaviors in videos, we propose a human action recognition approach to recognize several kinds of abnormal harmful actions in certain situations. This is a view-independent approach based on template matching. The template is composed of several action hyperspheres in subspace which encodes multi-view information of the actions. Recognition is then achieved by comparing the test action with the sample actions in the template. The action hypersphere contributes to view-independent action recognition, and the hypersphere-based recognition is superior to kNN classification in computation efficiency.The tiny face in surveillance video is an obstacle to face recognition. Therefore, we propose a two-phase face super-resolution approach. In the first phase, Locality Preserving Hallucination (LPH) algorithm is used to synthesize the global high-resolution face. In the second phase, we adopt neighbor reconstruction to synthesize the image residue and compensate the global face with detailed facial feature. Our approach can synthesize distinct high-resolution faces with various facial appearances efficiently, and this helps to eliminate the influences caused by tiny face.Then, image-based and video-based expression synthesis approaches are provided to tackle the problems in face recognition due to various facial expressions. The former uses Eigen-associative Learning algorithm to learn various facial expressions according to a face image with neutral expression. The latter is a two-level fusion approach which combines local linear and global nonlinear subspace learning. Amongst, the local linear subspace learning adopts eigen-representation technique for video sample compression in temporal domain; the global nonlinear subspace learning synthesizes optimized facial expressions in spatial domain. Synthesized facial expressions are close to ground truth expressions, and this improves the face recognition under various facial expressions.Finally, in order to diminish the influence of pose variation to face recognition, we introduce image-based and video-based 3D face reconstruction in turn. The image-based reconstruction firstly uses adaptive LLE to sample the training space in a nonparametric way, and reconstructs 3D face model based on the sampling result. Then constraint-based texture mapping is used to synthesize the realistic appearance. The video-based reconstruction adopts affine-rectified optical flow to track the feature points automatically aligned on the first frame of an uncalibrated monocular video sequence. Then SFM algorithm is used to recover the camera projection matrix and 3D coordinates of the facial feature points. We use facial feature points to deform a generic face model, and obtain personalized 3D face model and facial expressions.This thesis not only provides a holistic solution to intelligent video surveillance, but also explores some related key techniques and obtains primary results. In the conclusion of this thesis, we point out that there are still many open problems in intelligent video surveillance to be studied in depth.