Research On Human Body Complex Action Recognition Algorithm Based On Multivariate Feature Fusion

Action recognition is one of the main research contents in the field of computational vision.Accurate understanding of complex motion intention plays an important role in the fields of service robot,augmented reality and video surveillance.The existing technology has made great progress in bare hand motion recognition,but for the complex motion recognition of hand-held objects,due to the influence of occlusion,hand object joint and so on,it can not achieve the ideal effect.In order to solve this problem,based on RGB-D and sEMG data,this thesis studies the 3D skeleton feature,muscle state feature and interactive object feature of human body respectively,and on this basis,proposes a human complex action recognition algorithm based on multi feature fusion.The specific research contents are as follows:(1)A method of motion recognition based on multi-modal feature fusion of 3D skeleton and muscle state is proposed.For complex hand-held and non hand-held movements,skeleton VLAD and electromyogram VLAD features are extracted respectively,and then multi granularity features are constructed to express complex human movements,and multiple kernel learning method is used to fuse the two types of features and complete movement recognition.(2)An object recognition method based on shape feature is proposed.Firstly,66 dimensional features such as center of gravity,roundness and shape context are extracted from the contour of the object.Then,the feature selection method based on k-nearestneighbor,decision tree,probabilistic neural network,fuzzy rule and random forest is used to select the original feature set and obtain five diverse optimal feature subsets to train the diverse model.In the stage of local classifier fusion,five differential classification models are trained on five feature subsets to form local classifier set.In the stage of global classifier fusion,we use k-nearest neighbor,decision tree,probabilistic neural network,fuzzy rule,random forest,gradient lifting tree and support vector machine to build seven local sets and fuse the results of seven local sets to complete the task of object classification.(3)In this thesis,a multi-feature based hand object interaction recognition algorithm is proposed.Firstly,Kinect camera is used to obtain the color image data and depth image data of human body,and Openpose,the open source framework,is used to obtain the 3D coordinate data of bone points.Then,according to the distance between the hand and the head,the key frame is extracted automatically,and the relative distance feature of the key frame is extracted to describe the action,the Lop feature and the HSV color space feature are extracted to describe the object.Finally,the class feature is used to fuse and complete the complex action recognition task.