Research Of Hand Motion Detection And Robot Hand Imitation Based On RGB Images

Robot hand imitation has a wide range of important applications in rescue and disaster relief,underwater reconnaissance,and bomb removal.In the common imitation method of the robot hand,the method based on the data glove hinders operator's operation,and the method based on gesture recognition can only perform preset gestures.The method based on keypoint detection overcomes the above problems,is easy to operate,and can completely reproduce hand movements.This paper mainly studies a robot hand imitation method based on RGB image hand motion detection.Compared with depth-camera-based methods,it has a wider range of application scenarios.Compared with current RGB-image-based methods,it has good robustness in occlusion,multiple keypoints and other scenes.The research is divided into three parts: hand motion detection based on RGB images,binocular stereo vision based on keypoint energy map,finger joint motion solution and robot hand imitation.This paper first studies the detection of hand motion based on RGB images.This part can be divided into hand area detection and hand joint detection.In the hand area detection,this paper analyzes and selects the skin color detection method.For the problem that skin color detection is susceptible to skin-like background,this paper introduces motion information to eliminate the interference.In the object frame generation,this paper proposes a centroid-based regression box iterative algorithm.Compared with the contour detection method,it has better robustness and faster operation speed.In hand joint detection,this paper experimentally selects a suitable deep network method to replace the geometric method,which has better robustness to occlusion,multiple keypoints and other scenes.Then,this paper studies binocular stereo vision based on keypoint energy map.Due to the existence of keypoint detection noise,the three-dimensional coordinates of keypoints cannot be directly obtained in images after distortion correction and stereo correction.This paper proposes a keypoint correction algorithm based on energy maps,and verifies the feasibility through experiments.Next,this paper studies the solution of finger joint motion and robot hand imitation.According to the kinematic characteristics of human hand joints,the kinematics of the thumb and other four fingers are modeled separately,and the solution formulas for the rotation amount of each joint is given.Aiming at the inconsistency between the keypoint detection coordinate system and the kinematics modeling coordinate system,this paper proposes a "finger coordinate system" and provides a coordinate transformation method.Based on the kinematics solution of the human hand,this paper proposes a imitation method for commonly used underactuated manipulators,and verifies the feasibility by experiments.Finally,this paper verifies the proposed hand motion detection and robot hand imitation scheme,tests the success rate and speed,and analyzes the experimental results.