Discriminant Transfer Learning For Cross-view Action Recognition

Human action recognition in videos plays an important role in computer vision and pattern recognition due to its wide applications in human-computer interaction, smart surveillance, video retrieval, and visual reality. Traditional action recognition methods focus on a single view. They extract features and build models in a fixed view. However, in real applications, body appearances and motion trajectories of the same action can drastically vary from one view to another. Meanwhile, the corresponding data distributions and feature spaces are also different. As a result, action models learned in one view tend to be incapable of the recognition in another different view, which is a real challenge to traditional action recognition methods.This paper proposes a novel approach of cross-view action recognition, in which the samples from different views are represented by heterogeneous features with different dimensions. Inspired by transfer learning, we introduce a discriminative common feature space to bridge the source and target views. Two different projection matrices are learned to respectively map the action data from two different views into the common space by simultaneously maximizing the similarity of intra-class samples and minimizing the similarity of inter-class samples. In order to reduce the mismatch between data distributions of different views, an effective nonparametric criterion is added into the objective function. In addition, a valid locality constraint is also incorporated into the discriminant analysis to preserve the local manifold structure. This framework can be naturally generalized to the corresponding kernel version. The action samples are nonlinearly mapped into a high-dimensional kernel space, where the features are more discriminative. The proposed method is neither restricted to the corresponding action instances in the two views nor restricted to a specific type of feature. Experiments on the multi-view action datasets demonstrate the effectiveness of this method.Since single source view may provide partial action knowledge, an efficient multi-source transfer learning method is proposed to combine multiple source-view information. This method utilizes the labeled samples of all source views, integrates the explored information and transfers the learned knowledge to the target view, which can handle only a few or even no labeled samples available in the target view. The experimental results demonstrate that it is beneficial to fuse multiple source-view knowledge for improving the cross-view action recognition performance.