Research On Human Detection, Tracking And Action Recognition In Infrared Images

Recently,human detection, tracking and action recognition in thermal infrared imagery are very active in many fields, such as intelligent video surveillance, automatic vehicle driver assistance, and advanced human-machine interface. Compared with the visible light images, infrared images have almostly solved the problems of sudden illumination changes, shadows and poor night-time visibility in traditional computer vision fields, and furthermore have better performance on segmentation. However, the existing human motion analysis algorithms in infrared images have poor performance; especially the inherent characteristics of infrared images, such as the low contrast, low signal-to-noise ratio, uncalibrated white-black polarity changes, and the halo effect around the very hot or cold object, have made it a complex challenge for human detection, tracking and action recognition. In this paper, we aim at the study of key technology of thermal infrared imagery applied to the intelligent video surveillance systems. The principles of thermal infrared imagery and the characteristics of human target in infrared images are analyzed. Meanwhile, the human detection, tracking and action recognition algorithms are studied. The main contributions of the thesis can be concluded as follows.①In order to rapidly detect human targets in infrared image sequences, a hybrid classification features-based human detection algorithm is proposed. First, it uses the MAP-MRF model to locate the regions of interest (ROI). Then the method combines the pedestrian's shape-dependent and shape-independent features (including shape's morphological feature, inertia-based feature and histograms of oriented gradients (HOG) feature) to describe the ROI in the round, and last uses support vector machine (SVM) to classify and detect the pedestrian region. Experimental results using several long-wave infrared image sequences show the proposed scheme can detect pedestrians accurately by combining hybrid classification features, and can be employed in real-time applications.②In order to obtain more accurate target feature of human body in a single infrared image, a novel human detection algorithm based on double-density dual-tree complex wavelet transform (DD-DT CWT) of the wavelet entropy features is proposed. The DD-DT CWT combines the DT CWT and DD DWT and has the advantages of shift invariance, directional selectivity, freedom from aliasing, and a near-continuous wavelet transform. The proposed wavelet entropy properly reflects the energy distribution of the image in the frequency domain of the wavelet transform. The combination of DD-DT CWT and the wavelet entropy can thus describe the image features accurately. Experimental results have shown that our approach is very encouraging.③According to the problem of robust human tracking in thermal infrared imagery, a novel algorithm based on the intensity-distance projection space is proposed. The algorithm combines the intensity distribution and morphological characteristics of the human target. The method constructs the ROI's histogram representation in an intensity-distance projection space model. In addition, the tracking algorithm embeds the above mentioned representation model in the particle filter framework and updates the sample's representation model automatically. The experimental results show the proposed scheme performs more robust and stable than the classical tracking method.④In order to overcome the weak object feature description ability of a single locality preserving projections (LPP) for pedestrian target in infrared images, a co-occurrence matrix locality preserving projections (COMLPP) method is proposed to improve the robust performance of real-time pedestrian tracking in infrared image sequences. The co-occurrence matrices of the training set are first generated, then the feature representation vector of the locality preserving projections subspaces is generated by applying the locality preserving projection method to the projected samples of the training set on the co-occurrence matrix subspaces. At last, the above mentioned pedestrian representation model in infrared images is embedded in an improved mean shift based particle filter framework. Experimental results using different infrared image sequences show the proposed scheme achieves success in real-time pedestrian tracking system. Meanwhile, the proposed method can be adapted to complex and occluding scenes.⑤According to the problems of human action recognition in thermal infrared imagery, an infrared action database is constructed and a novel algorithm based on the fusion of human action silhouettes energy images and local scale-invariant features is proposed. The algorithm first makes use of the Gaussian mixture model and background subtraction to extract the human action silhouettes, while calculating the energy images for the action sequences. Then, the scale-invariant 3D Harris corner detector and brightness gradient cuboids descriptor are applied to obtain the local scale-invariant features from the action sequences. Finally, the human action is represented by fusing the energy images and local scale-invariant features, and recognized by using the nearest neighbor classifier. Experimental results show that the results of the proposed method are very promising.The above creative works in the thesis, covering the main areas of intelligent video surveillance tasks which including human target detection, tracking and action recognition, from low level to high level technology, provide a theoretical support for the applications of thermal infrared imagery in video surveillance systems.