| The action recognition is an important research topic in the field of computer vision.Different from image data,the variable and complex information from time domain and space domain in video actions make it more difficult to efficiently and stably extract discriminate action representations from complex action interaction scenes.Hence,the researches on video action recognition based on deep learning are studied as follows:1.In view of the lack of fine-frained action representation based on 2D convolutional neural network(2D CNN),a spatial-temporal dual attention model is studied in this thesis,in which the semantics of multiple visual perception layers of convolutional neural network are extracted to enrich the action representation level.Besides,considering the fully-connected long short memory(FC-LSTM)network in the CNN-LSTM model is prone to lose spatial feature information,and weak in selecting key feature information,different types of LSTM units are used to model the temporal sequence features with different dimensions in our proposed method.Then,in order to strengthen concentration on key action information,both temporal attention module(TAM)and joint spatial-temporal attention module(JSTAM)are built.Finally,after dimensions reduction by PCA methods,the spatial-temporal feature descriptors are fused to weigh the recognition contribution brought from different attention networks.At the same time,the end-to-end training of spatial-temporal dual attention network(STDAN)is realized.Experimentally,the comparison results with state-of-the-art methods on multiple action recognition datasets have shown the superiority of our model,and the visual analysis also proves that the model can effectively extract key features.2.Considering that the shallow nature of convolution network 3D(C3D)model is not conducive to the deep representation of the action law even affects the recognition accuracy.Based on C3 D model,this thesis proposes a residual attention network model.In proposed model,aimed at reducing the inefficient computing of 3D convolution operation,the 3D convolution kernels are decomposed along the spatio-temporal dimension,then a 2D spatial kernel and a multi-scale 1D time kernel are obtained to ease the training burden.In addition,through introducing the channel attention andspatial attention mechanism,the model is endowed with flexible ability in feature selection.Then,the residual attention network is constructed by stacking each residual block.In this basis,a two-stream action recognition model is constructed based on the designed residual network.To deal with the weak interaction of spatial-temporal features,multi-stage fusion strategy is used to fully explore the two-stream model.Finally,PCA is used to implement the dimension reduction of fused feature descriptor,which is classified by multi-task SVM for action recognition.The experimental results show that the designed residual attention network model can achieve higher recognition accuracy and balance the training efficiency and trainable parameters.Besides,the two-stream model in the real dataset UCF101 has made further performance improvement,which proves the feasibility of the model. |
