Research On Multi-modal Based Human Pose Recovery And Similarity Evaluation System

Thanks to the development of low cost depth camera like Kinect, the motion capture technique has been promoted a lot. This technique is changing people’s life by entering new fields like home entertainment, education and HCI. In this paper, we discuss the current status of human3D pose recovery and pose similarity evaluation methods. We also implement a pose recovery system and pose similarity evaluation system based on our method.This paper describes a multi-modal based human pose recovery system. It combines depth image and RGB image as input, using foreground detection, face detection and color tracking algorithm to label body parts in the image. Then we compute the position of joints to reconstruct the human skeleton and apply the post-processing on the reconstructed human pose.This paper also describes a pose similarity evaluation system which can give a normalized score. We use relational geometric features which are selected by Adaboost algorithm to represent the motion data. Then we use Open End Dynamic Time Warping algorithm to align two sequences. After that, we can get a correlation based score using quaternion representation of the poses. This score can represent the similarity objectively.We also implement a dance training system using our similarity evaluation method. Dancers can see the avatar while dancing and get a score after dancing. This system makes dance learning an interesting event and can be applied at home.