Learning Coordinated Motions Of Dual Arm-Hand For Signing Gestures

Nowadays,service robots communicate and interact with people mainly through vocal,visual,or verbal means,which is quite convenient for normal people,but they cannot fulfill the requirements of the hearing-impaired.In order for this group of people to enjoy the convenience that technological advancements bring,the research and development of sign language robots should be conducted,so as to enable a more suitable way of communication via sign language.Most of the previously proposed sign language robots are programmed with sign language motions through tedious and inefficient manual programming or kinesthetic teaching.Thus,guided by the idea of Imitation Learning and Motion Retargeting,this thesis conducts research on motion retargeting for sign language motions to accomplish an easy-to-use approach of equipping robots with the ability to perform signing.In this thesis,a model of dual arm/hand movements is proposed for describing complex sign language motions,based on which a Sign Language Demonstration Capture and Acquisition System is constructed to record human demonstrated sign language motions and build a Chinese Sign Language(CSL)demonstration dataset.Since previously proposed motion retargeting methods mainly deal with large arm movements and do not perform well on arm movements within a small region,the major focus of this thesis is on learning the complex coordinated motions of dual arm/hand within a small region.The main research contents and contributions of this thesis are as follows.(1)A generic sign language motion model that well captures anthropomorphism is proposed,based on which the corresponding Sign Language Demonstration Capture and Acquisition System is built and used to construct a Chinese Sign Language demonstration dataset.Taking into account of anthropomorphism,adaptability and characteristics of sign language motions,this thesis proposes a model that fully captures the motions of arms and hands,and builds a demonstration data capture system,using an optical motion capture system,a pair of data gloves and an USB camera to obtain arm/hand motions as well as video information,which are then synchronized in real-time.With the use of the system,a Chinese Sign Language demonstration dataset is constructed,which consists of motion data of 208 sign language words and 20 sentences.(2)A motion retargeting method based on motion keypoints and position scaling is proposed.To cope with the difference in size between humans and robots,this thesis implements a position scaling-based motion retargeting method and analyzes its deficiency through experiments on sign language motions.Considering the importance of relative motion relationship between body parts for correctly conveying information via sign language,this thesis introduces the concept of motion keypoints into motion retargeting by integrating relative pose constraints into the optimization problem.The experimental results indicate that the proposed method resolves the position misalignment problem encountered by ordinary position scaling approaches,and the average absolute error of every dimension of the relative position trajectory is reduced by 23.5%?88.7%,which validates the method's ability to preserve relative motion between left and right wrists.(3)A leader-follower Dynamic Movement Primitives(DMPs)based motion retargeting method is proposed.This thesis utilizes the powerful generalization ability of DMP models,by learning DMPs in a leader-follower manner from human demonstrations,to preserve the original path shape,motion rhythm and relative motion between body parts in the retargeted motions.Based on this model,graph optimization is employed to optimize DMPs' start and goal positions,as well as robot joint trajectories while considering the robot's feasibility constraints and collision avoidance,to find a new trajectory suitable for robot execution and deviating from the original trajectory as few as possible.The proposed method is compared with other commonly used motion retargeting techniques by conducting experiments on a subset of CSL words.The experimental results indicate that the success rate of the proposed method on the test motions is 100.0%.Among the trajectories of all the feasible motions retargeted by the proposed method,100.0%of the orientation trajectories,88.9%of the relative position trajectories and 66.7%of the absolute position trajectories yield smaller Frechet distance values than their counterparts belonging to the other method,which are reduced by 94.9%?99.7%,8.6%?70.4%and 4.8%?72.5%respectively,thus validating the proposed method's ability to maintain motion similarity and preserve relative motion relationship.