Research On Human Action Recognition And Detection Based On Multilayer Visual Features

Vision based human action recognition and detection method is the basis of object behavior understanding,which is of great significance for improving the ability of manmachine cooperation and intelligence of intelligent unmanned system.The traditional action recognition methods mainly depend on manual-designed features,and theses features can only represent shallow local information,lacking the ability to represent global semantics information and temporal sequential information.This thesis researches on action recognition and detection method based on multi-layer visual features,and focuses on making full use of the learning ability of convolutional networks to enhance the action recognition and detection accuracy and efficiency,while fusing the spatial information,temporal information and semantic information.And the main work and contributions of this thesis are as follows:1.We improved an algorithm of human key points detection based on convolutional neural network.By extending the network layers,the algorithm improves the positioning accuracy of the human key points,and verifies the performance through the tests on the Penn database.Furthermore,the key points are connected with the key point affinity field,and the multi-object poses are estimated.Based on the poses estimated,an initialization method for tracking object in ”Man-Vehicle Following” system is proposed.The validity of the proposed method is verified by tests on the data collected by real vehicle.2.We proposed a new method of human action recognition based on fusing multilayer features.The proposed method focuses on the clipped video clips,and combines a new feature fusion and feature coding layer as the core to fuse the apparent information and the inter frame motion flow field information of the continuous multi frame images.The feature vectors are encoded by the pose trajectory proposed in this paper,and then the video representation feature vectors for classification are calculated,and the SVM is used to classify the features.Experiments on open source databases such as KTH database,Penn database,HMDB-51 database and UCF-101 database verify the effectiveness of the proposed method.3.We proposed a novel convolution method,and use this convolution method to improve a three-dimensional convolutional network structure.And then,an action detection framework of multilayer network is used to detect the action clips in temporal domain.Furthermore,combining the inter frame motion flow field of the detected video,the input data is enhanced to further improve the accuracy of the detection algorithm.The validity of the improved method is verified by experiments on the THOMUS-2014 open source database.