Research On Action Recognition Based On Infrared Video Analysis

Action recognition is an important task in computer vision, and has a wide application in the future. Basically, most of the current research efforts for action recognition have been put into visible imaging videos. Few people make full use of infrared video for recognition task. To address this problem, this thesis is developed for action recognition upon infrared video. Firstly, a new infrared dataset is constructed. Then feature evaluation work is executed on this dataset with various action descriptors. Finally, an improved two-stream Convolutional Neural Network(CNN) method for infrared action recognition is proposed. The detailed work is as follows:The investigation is firstly executed on current visible spectrum action dataset. Inspired by the construction approach of existing visible datasets, the thesis has built a new Infrared Action Recognition(InfAR) dataset. All infrared samples are sampled from real-world varying scenes. InfAR dataset collects 12 common human actions from infrared video and the imaging factors conclude background, occlusion, viewpoint and season, and so forth.Based on constructed Inf AR dataset, feature evaluations are carried out in three aspects including low-level descriptors, feature fusion and imaging factor. Ten different low-level local descriptors are extracted from infrared video. Then they are encoded with three encoding methods and tested with two kernel functions. The experimental results show the best performance for dense trajectory feature on the InfAR dataset is 68% and the worst performance for HOG feature is 26%. The thesis considers 5 different representative descriptors in the early and late fusion evaluation. The experiments reveal the late fusion facilitates a better performance than early fusion and the number of features seems not to determine the final performance. Besides, this thesis also evaluates two pairs of imaging factors(“simple background/complex background”, “summer/winter”) on InfAR dataset. The results reveal imaging factors make an important effect on the final performance.Aiming at the problems of the low performance of hand-craft feature and lacking of texture for infrared video, the thesis proposed an improved two-stream CNN method for infrared action recognition, which combines the appearance information from appearance stream and motion information from motion stream. Finally, actions are recognized using the SVM classifier. Experimental results show that this method can effectively recognize actions in infrared videos and has strong robustness.