Research On Human Action Recognition Based On Deep Learning

The exponential growth of information puts forward higher requirements for data processing and analysis,among which,the growth of action data is particularly considerable,which also promotes the industrialization of action related learning tasks such as action detection,action retrieval and action recognition.The application prospect of action recognition in the fields of human-computer interaction,security monitoring,dynamic games and other fields have received special attention from researchers.Moreover,with the rapid development of hardware and sensor technology,it is easier to obtain various action data and the rapid analysis of action data is more convenient.Action recognition based on various kinds of modality data has been well developed and explored,among them,the skeletal action sequence win even greater favor from action recognition research scholars for its concise representation of action and its insensitivity to illumination and background.However,most of the existing studies based on skeletal sequence start with extracting holistic motion feature or supplement them with local features,and thereon to identify action,which makes the detailed features of action easier to be neglected.This thesis starts from another perspective by fusing local motion features to achieve the purpose of action recognition.The main work of action recognition based on skeletal action sequence with deep learning in this thesis is as follows:(1)Data preprocessing and motion representation transformation of skeletal action sequences;the preprocessing of the data includes transforming the Cartesian coordinate of original representation into cylindrical coordinate and normalizing the temporal duration of the skeletal motion sequences.The transformation of motion representation is to transform the global joint motion representation into the local motion representation based part joint,which includes two stages: partitioning part joints and calculating new motion representation.(2)Constructing a network model based on the new motion representation for action recognition,and explore the action recognition method based on the fusion of local part motion features;LSTM and CNN are combined as the basic layers of the network to extract the spatial and temporal features of action.The multi-stream network based on the LSTM-CNN layer is used to extract the local motion features of each part,and the strategy of score fusion is employed to predict the action class.Experiments implemented on two classical datasets verified the performance of the model for action recognition.(3)In order to improve the identification accuracy of the model and reduce the mean error,the residual network structure and spatial configuration items are introduced to optimize the model;in multi-stream network,the identity mapping is adopted to provide shortcut connection for the input and output of LSTM-CNN layer,which extends the LSTM-CNN layer into residual block.Label mapping for spatial configuration amplify the motions of joints in different degree.(4)Comparing the proposed method of action recognition with other approaches based on CNN and RNN,and the experimental result shows that the method proposed in this thesis is superior to some of them and is comparable with those with high accuracy in recognition.