Research On Foreground Extraction And Action Recognition For Intelligent Video Surveillance

With the rapid development of the society, many security issues and terroristic events show in front of us, bringing greater challenges to the management of society. Toward protecting public security and responding quickly to emergencies, people need a more powerful method to gather and utilize information. Fortunately, the fast evolution of computer and network technology has made large-scale video surveillance possible, making it a new infra-structure for modern cities. But, the amount of information grows explosively during the development of large-scale surveillance systems, while traditional human-based monitoring cannot suffice. There is a common understanding in both security department and research bodies that it is urgent to develop a new generation of computer technology to analyze, recognize and index videos automatically.Foreground extraction and human action recognition are core issues in intelligent video surveillance. However, the degree of freedom of human body is large, and variations among appearances and styles of individuals are complex, causing many difficulties in analyzing them accurately. In this thesis, we focus on three key problems, i.e., moving foreground detection, deblurring of moving foreground and human action recognition. The main contents of the thesis are as follows:First, we introduce some basic concepts about foreground extraction and human action recognition, and do surveys for the three key problems introduced above. The terminology of the thesis is defined for clarity.Second, traditional background modeling methods paid little attention on the information redundancy inside small space-time local regions, causing their models to be quite bloated. To solve this, we propose a block-based background model, using nearing pixels in the same block to reduce redundancies. Also, we simplified the color codebook, so the searching speed improves considerably.Third, in the literature of image deblurring methods, most of them targeted at blur caused by camera shaking. In video surveillance, blur is commonly caused by object motion instead. In this thesis, we propose a motion deblurring method for moving foreground objects in the video. We use a affine motion model, and estimate its parameters using KLT feature tracker. After that, the alpha-matte of the object is synthesized using the estimated point-spreading-function, without user intervention. We also proposed an iterative Richard-Lucy deconvolution method to handle spatially variant motion blur.Finally, we investigate into human action recognition, and propose a novel framework, which combines local features and manifold learning technologies. In this new framework, space-time interest points are used as feature descriptors. Based on Manifold Elastic Net, we proposed a multi-mode learning method called Elastic Manifold Embedding. Training samples are first clustered into sub-classes, and then projected onto a manifold to reduce dimensionality. The result of the framework outperforms many state-of-art methods.