Video-based Action Recognition

Video-based action recognition is one of the most challenging problems in the field of computer vision.It is such a process in which a computer can automatically analyze and identify the action category from the video which contains human behavior.Video action recognition has important application value in many fields,such as intelligent video surveillance,video retrieval,human-computer interaction and so on.In this thesis,we study video action recognition based on convolutional neural network(CNN).The main work and contributions are as follows:(1)Video Action Recognition based on Segmented Two-Stream Convolution Neural Networks.By replacing Alex Net-based CNN with GoogLeNet-based CNN,and applying BN-Inception models instead of Inception V1 models in GoogLeNet-based network,the number of layers in two-stream CNN is increased,and the feature expression ability is therefore enhenced.Furthermore,different levels of features from the above CNN model are integrated together in order to achieve complementary information on the behavior of samples in different degrees.To obtain a better generalization ability of the proposed model,the parameters of the CNN are initialized based on pretrained network model.Then these parameters are further adjusted based on Error-Back Propagation algorithm.In order to achieve a more effective representation of the local temporal structure of video behavior,we construct the segmented two-stream convolution neural networks in which the two strams are composed of spatial stream and temporal stream.The experiments based on UCF101 and HMDB51 demonstrate the effectiveness of the proposed algorithm.(2)Video Action Recognition based on Ensemble Learning.To furtherly improve the performance of action recognition,this thesis also studies ensemble learning based methods.For decision-level ensembling,several methods based on weighted voting strategy are proopsed based on learned individual action recogniton models.Experiments on UCF101 and HMDB51 show that the recognition performance can be further improved by ensemble learning.For feature-level integration,the multi-stage leraning based feature fusion method based on the segmented double convolution neural network is also studied.