Vision Based Human Pose And Hand Gesture Description And Recognition

Human pose recognition is one of the important topics in the field of computer vision,which contributes to achieve efficient human-computer interaction and is highly concerned by researchers.Hand gestures are details of human body and are often studied separately from human poses.In gesture recognition,effective gesture description is very important.Based on the visual information of human gestures,this paper researches dynamic motion description methods,static posture description methods,hand segmentation methods,3D hand gesture description methods,gesture recognition methods,and their application values.Firstly,a hand gesture segmentation method based on depth information is proposed.By setting an adaptive depth threshold and finding suitable contour inflection points,the hand region can be accurately segmented from the depth map.In addition,a platform for real-time acquisition of human skeleton and hand gesture information is built based on the Kinect sensor.The platform can display and save color images,depth maps,human skeleton,and segmented hand gestures in real time,which provides accurate data for subsequent gesture description.In order to capture the general description of the gesture motion,a gesture description and recognition method based on motion primitives is proposed.The motions of each joint between adjacent frames are calculated to represent its 3D trajectory.The low-level features are extracted to improve the efficiency of the method without the loss of motion information.The low-level features are clustered by K-means algorithm to obtain motion primitives,which are quantized by histograms.Additionally,the distances between the key skeleton joints are calculated to represent the static posture,which is a good complement to the motion primitive histogram.Their combination forms the whole gesture descriptor which is recognized by the Random Forest algorithm.The experimental results verify that the gesture descriptor is invariant to the speed of action,and the gesture recognition method achieves excellent results on the benchmark action datasets.Considering the limitations of the hard-clustering method,a soft quantitative learning method is proposed for quantifying and representing gestures.A spatio-temporal multi-scale soft quantization network is proposed,which is a trainable soft quantization method using RBF neurons.In different spatial levels,including the human posture level,body part level,and skeleton joint level,different RBF neuron groups are used and the RBF neuron groups within each level are not shared.In order to capture the temporal information of gesture,the motion features are quantized on different temporal scales where the RBF neuron groups are shared.Additionally,the spatio-temporal multi-scale soft quantization network is an end-to-end neural network,which can be effectively supervised by labels and trained by back propagation and gradient descent methods to realize gesture recognition.The experimental results verify the effectiveness of the soft quantification method and show that the spatio-temporal multi-scale description is meaningful.At last,the description and recognition of hand gestures are studied especially and a 3D hand gesture description method is proposed.Both the local and global 3D shape information in multiple scales are sufficiently utilized in the feature extraction and representation.The 3D depth context information of hand gestures is extracted at salient feature points to obtain discriminative pattern of hand shape.The DTW algorithm is improved with Chi-square Coefficient method,where the Euclidean distance is replaced in calculating the similarity between hand gestures.This method is invariant to geometric transformations and nonlinear deformations,and is robust to noise and cluttered background.It achieves state-of-the-art performances on benchmark hand gesture datasets and its efficiency is fast enough for real-time applications.