Research On The Co-Learning On Multi-Hypergraph Based Human Action Recognition

Vision-based action recognition has always been a hot topic in the field of computer vision.Early action recognition methods are based on color images.When the lighting conditions change,the quality of color images will be affected,and the accuracy of action recognition will also decrease.Compared to color images,depth maps are insensitive to changes of lighting conditions.They can also provide additional 3D geometric information,which is particularly important for the extraction of action features.In recent years,people have designed or learned many excellent features based on depth information.There are correlation and complementarity between them.Good fusion of multiple action features can effectively improve the accuracy of action recognition.Hypergraph has its unique advantages in characterizing the high-order correlation between data.Therefore,we propose a colearning method on multi-hypergraph(MHCL)to solve the problem of multi-feature fusion in depth-based action recognition.Firstly,using the action sequences as the vertices of the hypergraph,we propose a belief propagation based hyperedge construction method.Specifically,we take each vertex of the hypergraph as the central vertex of current hyperedge in the first step,and construct a global energy function based on the distance relationship between all the vertices.Then,we determine which vertices belong to current hyperedge by minimizing the global energy function.The hyperedges constructed in this way are flexible in size,and the characterization of high-order correlation between vertices is more accurate.Secondly,a co-learning method on hypergraph is proposed.We optimize weight of hyperedge while learning the category of action sequences.Furthermore,the co-learning on hypergraph is extended to co-learning on multi-hypergraph.The weights of hyperedges and hypergraphs are optimized simultaneously while learning the category of action sequences.So the multi-hypergraph fusion is performed on the optimal hypergraph structure,and the purpose of multi-feature fusion is also well realized.In order to verify the performance of the proposed algorithm,we have conducted experiments on four public action recognition datasets,which are MSRGesture3 D dataset,MSRAction3 D dataset,MSRActionPairs dataset and the large-scale NTU RGB+D dataset.Compared with other state-of-the-art methods,the proposed MHCL method achieves better performance.For the two key points in our proposed MHCL method,the belief propagation based hyperedge construction method and co-learning on hypergraph for optimizing the weights of hyperedges,we also verify their respective contributions through experiments.At last,we also discuss the key parameters in our proposed MHCL method in detail.