The Study Of Vision-based 3D Hand Gesture Interaction

The birth and use of Motion Sensing technology is a new revolution in the human-computer interaction field,followed by a genuinely new way for a computer to make some sense of the world around it and various innovative HCI applications.Since the launch of the Mircosoft Kinect, it has evoked great repercussions in academia and makes 3D Human Computer Interaction becomes an active area of research. In this paper,the 3D dynamic gesture recogniton using Kinect is studied and its main idea is as follows:First, a novel method called Multi-Dimensional Weighted Dynamic Time Warping(MDW-DTW) is proposed for recognizing hand gestures using a sequence of real-time skeleton image data acquired by Microsoft Kinect sensor. Six 3D body joint motion trajectories are obtained, giving us multi-dimensional time series for each gesture sample. In order to achieve a better alignment,the proposed MDW-DTW is employed,which is an extension of DTW to handle multi-dimensional time series alignment problem. Its main idea is that the time series point at each time instance is considered as a basic processing unit and also both Dimension Weighting scheme and Feature Weighting scheme are incorporated into the DTW framework when computing the cost matrix. Experimental results demonstrate that MDW-DTW outperforms DTW and improved algorithms based on DTW.Second,a 3D dynamic hand gesture interaction system with multiple functions is developed.It provides gesture recording,gesture management,gesture recogniton and some other functions.Moreover,we can specify the control command for each gesture to operate Windows software with shortcuts.The developed system is simple, convenient and easy extensibility.Finally, for our example application,a simple photo album browser system based on gesture control is designed and realized to let user experience the feeling of virtual touch using Kinect and control images at a distance. After several tests,it turns out that our recognition algorithm can meet the real-time capability and robustness which a interaction system requires.Besides, the designed interactive gestures are also able to meet the user expectations.