Research On Open Domain Action Recognition Algorithm Based On Cross-Domain Learning And Double-chain Fusion Network

In recent years,human action recognition has received more and more attention in the fields of computer vision and machine learning.Although many related action recognition algorithms have been proposed,these methods often assume that data originates from the same domain,action features are extracted in the same scene,and train a reliable model when the training samples are sufficient.However,in the real application scenario,the training data with labels are very small,and due to the change of camera angle and position,the human body shape and trajectory of the same action will change significantly,and the feature space and corresponding data distribution will also change.Therefore,cross-domain action recognition in multiple domains is a challenging topic.Based on this development trend,the research work of this paper mainly includes the following three parts: 1)Constructing multiview multi-modality human action dataset(referred to as MMA);2)Exploring open domain action recognition based on feature learning and cross-domain learning;3)An open domain action recognition algorithm for pairwise fusion networks is proposed.The specific work is:1)Constructing a multi-view multi-modality human action dataset(referred to as MMA).The number of action categories,samples,camera views,and scenes captured in most existing action datasets is often limited.In addition,these datasets can only be used for a single learning task,such as single-view learning,cross-view learning,and multi-task learning,which is not conducive to subsequent action recognition research.Therefore,a multi-view multi-modality human action dataset is constructed.The dataset contains a total of 7080 action samples,which are from two scenes,each scene includes three views.In addition,these samples contain 25 action categories.It is divided into 15 single action and 10 double action.In order to fully evaluate the dataset,it was experimented with different tasks.The experimental results show that the MMA dataset is challenging for these three learning problems due to significant intra-class changes,occlusion problems,changes in view and scene,and similarity of multiple action categories.2)Exploring the open domain action recognition based on feature learning and crossdomain learning.The action recognition problem of open domain is discussed in three different ways: 1)Feature learning: extracting hand-crafted features and deep learning features from video,and then evaluating and discussing their performance in controlled and uncontrolled environment respectively.2)Unsupervised cross-domain learning: because it is difficult to obtain labeled samples in the target domain,unsupervised cross-domain learning algorithms can be used for action recognition;3)Supervised cross-domain learning: if target domain has some labeled samples,but the number of them is very limited,then supervised cross-domain learning methods will be a good choice,so the six algorithms of supervised cross-domain learning are also evaluated on the same dataset.In addition,the cross-domain learning problem on the MMA dataset was further explored.3)Proposing pairwise fusion network for open domain action recognition(PFN).In this algorithm,an end-to-end double-chain network structure is proposed.It can jointly fuse different spatiotemporal features from the video,learn domain invariant feature for the source and target domain,and build the classification model.To model the shift from the source domain to the target domain,the parameters in the corresponding layers of the PFN are asked to be relevant,but not exactly identical.Because the number of existing action data samples is small,the training of the network is not enough.Therefore,in order to increase the number of training samples,the paired samples of the source domain and the target domain are constructed,thereby directly increasing the number of training samples of the network.Through a lot of experiments on two different action datasets MMA and ODAR,our PFN algorithm has better performance in cross-domain action recognition.