Static Gesture And Continuous Actions Of Upper Limb Recognition Based On KINECT

With the development of computer technology, non-contact interaction which is natural and simple becomes more and more popular. Gestures and actions, as common means of daily communication, have become the main ways of non-contact interaction. Now the recognition algorithms based on monocular vision are vulnerable to environmental factors such as illumination, complex background and so on. Although the recognition based on binocular vision can largely avoid these effects, the calculation process is more complex. This situation can be improved with the appearance of depth camera such as KINECT which could provide both color and depth information about the whole scene, the foreground and background could be extracted effectively using the help of the motion sensing camera. Based on the depth data provided by KINECT, intense research has been made on gesture recognition and action recognition in this paper, the contents are as follows.First, a gesture recognition algorithm based on depth information is proposed. A binarization image of hand is generated through segmenting the depth map with the skeleton data provided by KINECT. Then the center of palm could be located while finding the maximum inscribed circle and the fingers could be detected by shape descriptor. Finally gesture feature extracted according to the hand information is used to train a random forest model. The followed experimental results show that the accuracy of the method.Second, in order to recognize the continuous actions of upper limb from depth video a dynamic programming method is put forward. To begin with, a normalized skeleton is extracted from the depth map to represent the human body. Then, each action of the human body is modeled by hidden Markov model, and the action sequence is divided into multiple sub sequences by prior knowledge. Finally, a cost function is generated according to hidden Markov models, the dynamic programming algorithm and a threshold model are used to get the optimal action identification tag. Experimental results about 20 groups of test data show that the algorithm can effectively identify the continuous actions.