Multi-Task Human-robot Interaction System Based On Human Pose Estimation

In recent year,development of traditional industrial robots encounters a bottleneck.Seeking for further improvement of productivity,a flexible way that human and robots work together to finish certain tasks is being considered.In this paper,a vision-based human-robot interaction system that satisfies multiple tasks is proposed,varies aspects that composite the system is studied.Firstly,to solve the problem of noise measurement sensitiveness and information lacking of pose estimation based on single type of sensor,a 3D human pose estimation method based on multi-sensor fusion is proposed.According to properties of each sensor measurements,an target function is established.2D visual information detected by multiple cameras and limb pose data detected by wearable IMUs are fused by optimizing the energy function to estimate the human pose.Secondly,when dealing with multi-human pose reconstruction under multicamera scenario,problem to identity pose of the same person across different views should be addressed.An iterative pose pairing method based on matching rate is proposed to unify the pose ID under multiple views.An each-iteration-check step is applied in this method to prevent possible false pairing by adjusting the iteration priority when overlapping of subjects happens.A pose tracking method is applied to keep consistent ID of 3D poses detected under continuous time sequence.The backward search feature of the method allows pose that not appears in every frame being tracked.Finally,under human-robot interaction environment,it is common that robot has different task assigned under certain situation.A robot action pattern based on target generating is proposed so that the robot could swiftly switch among the tasks by changing the target generating method.Accordingly,a model predictive control method is designed so that the robot could adjust the parameter to adapt various control requirements under different tasks.According to each aspect of the research,experiments are designed respectively.Pose estimation experiments under different sensor configuration cases show the result under multi-sensor fusion achieves higher accuracy and robustness under partially occluded visual information.Based on the pose dataset captured,pose pairing experiment is performed and the result shows that method proposed in this paper has better correct pairing rate in comparison with state-of-arts.Target tracking experiment is performed to test the smoothness of the model predictive based control method under different parameter.Multi-person human-robot safety experiment and several humanrobot interaction experiments are also designed to verify the feasibility of the system under various environments.The experiment results show the system is adaptable to multiple tasks,providing a reliable solution for human-robot cooperation.