Human Action Recognition Algorithm Based On Multi-Task Learning

Human action recognition has been one of the most important research topics in the field of computer vision and pattern recognition.It has extremely high commercial value and wide application prospect,such as human-computer interaction,intelligent video monitoring,intelligent video retrieval,intelligent driving,etc.The majority of existing action recognition methods mainly focused on video recorded in controlled environment.However,due to the influence of factors such as camera movement,background occlusion,angle change and illumination change and so on,these methods are hard to deal with video taken in real world and cannot meet the actual demand.It is urgent to propose a feasible method of human action recognition.At present,methods of human action recognition are mainly single-task learning,which consider the action categories in the dataset as independent tasks,and train classifier separately for each action category ignoring the inner relationships between them.In real world,there are always some relationships between action categories.This paper applies the multi-task learning method in machine learning to recognize human action.Multi-task learning can effectively improve the performance of the algorithm by learning the shared information between multiple tasks simultaneously.However,there are strong correlations or weak correlations between action categories,and learn all the action categories together would not make it the best.Therefore,this paper combines multi-task learning with group information,which assigns action categories with strong correlations to the same group and action categories with weak correlations to the different groups.At the same time,this paper exploits multi-task learning method with group information to learn the shared information and the difference information between groups to improve human action recognition performance.The main contents of this paper are as follows:(1)Extract improved dense trajectory features.Firstly,pre-process the video by eliminating the impact of the camera motion.Then,video frames are dense sampled in multiple-scale,the trajectory is obtained by tracking feature point,and HOG,HOF and MBH are used as local descriptors to describe this trajectory.Finally,due to different video has different number of trajectories,this paper exploit Fisher Vector to encode these trajectories into a fixed-length vector as the feature vector of the video representation.(2)Pre-group the action categories.Given Fisher Vector is constituted by the Gaussian Mixture Model and different action category has different Gaussian components,this paper exploits mutual information to measure the relationship between Gaussian components and action categories.Moreover,according to the principle that the more same Gaussian components that action categories have,the more similar they are,calculate the similarity matrix of action categories.Finally,this paper uses Affinity Propagation algorithm to group action categories with strong correlations into the same group and action categories with weak correlations into different groups.(3)Multi-task learning based on grouping information.Regarding each action category as a task,first multi-task learning for human action recognition within the same group and learn the shared information.Then multi-task learning within all action categories and learn the different information.This method takes into consideration the relevance between action categories and the difference between action categories,and provides the guarantee for improving the performance of human action recognition.In order to verify the effectiveness of the proposed algorithm,we validate the two standard datasets on HMDB51 and UCF50 respectively.The experimental results show the robustness and excellent performance of the proposed human action recognition algorithm based on multi-task learning with group information.