Research On Two-stream Convolutional Neural Network Fine-tuning Algorithm For Action Recognition

With the remarkable achievements of deep learning in image classification,many scholars have also used deep learning for video classification.As the action recognition has been widely used in the fields of human-computer intelligent interaction,automatic retrieval,video monitoring,automatic labeling,medical diagnosis and so on,the significance and broad prospects of this technology make it a research hotspot of computer vision.As a representative method of deep learning,convolutional neural networks(CNN)have been widely used in the field of action recognition in recent years.There are currently two main methods for action recognition task : 3D CNN and two-stream CNN algorithms.In this paper,spatial stream and temporal stream are trained separately and a two-stream 3D convolutional fusion neural network algorithm is implemented.Experimental results prove that the the two-stream 3D convolutional fusion neural network is effective.Based on the fact that the correntropy is robust to impulsive noise and negative samples,instead of the softmax loss,this paper combines the correntropy and the two-stream 3D convolutional fusion neural network and proposes a two-stream CNN fine-tuning algorithm based on the versoria loss function.Experimental results prove that the two-stream CNN fine-tuning algorithm based on the versoria loss function on action data sets is effective and robust under impulsive noise.Due to the similarities between actions,this paper adds fisher discriminative criterion to the loss function of two-stream CNN which makes the features extracted by the two-stream network have smaller intra-class divergence and larger inter-class divergence.This paper adopts a kind of learning update method for the parameters of fisher discriminative regularization,which is similar to the stochastic gradient descent algorithm and is more suitable for network training.Experimental results prove that the two-stream CNN fine-tuning algorithm based on fisher discriminative criterion is effective.To make the output of classifier more similar to the expected output,this paper proposes a two-stream CNN fine-tuning algorithm based on the LMNN(Large margin nearest neighbor)constraint to maximize the prediction values of true class and increase the gap between prediction values of true and false class.Experimental results prove that the two-stream CNN fine-tuning algorithm based on LMNN constrain is effective.