Zero Shot Action Recognition With Local Preserving Canonical Correlation Analysis

Automatically recognizing a large number of action categories from videos is of significant importance for video understanding.Most existing works focused on the design of more discriminative feature representation,and have achieved promising results when the positive samples are enough.However,very limited efforts were spent on recognizing a novel action without any positive exemplars,which is often the case in the real settings due to the large amount of action classes.In practice,not all the action categories can be enumerated and duo to the limitation of human and material resources,in some cases,a sufficient number of samples to train a good model cannot be collected and labeled.In addition,the existing methods can only recognize the pre-set action category,when faced with an unknown category sample,they cannot make the correct prediction while classifying the unknown sample of as a known category.And when a new category needs to be recognized,the entire model should be retrained.Zero shot learning is an attractive approach aiming at handling the difficulty of collecting ever more data and labeling them exhaustively.This paper proposes a ZSL-based action recognition method with the idea of local preserving canonical correlation analysis.Specifically,it constructs a mapping from visual and side information to a common CCA feature space,using a manifold-regularized term.The impact of domain shift is also taken into consideration.Approaches of self-training and hubness correction are applied to improve the robustness of this method.The proposed method is evaluated extensively on popular human action datasets of HMDB51 and UCF101.The results demonstrate that the proposed method achieves a better performance in the tested dataset against the state-of-the-arts with a simple and efficient pipeline.