Human Hand Detection And Tracking System For Cooperative Robots

With the development and popularization of intelligent robots,research on the interaction between humans and robots has become increasingly important.The hand,as the most convenient and efficient interaction channel between collaborative robots and the human body,is one of the main ways of human-machine interaction.Hand detection and tracking are important steps for collaborative robots to achieve human-machine interaction.In recent years,progress has been made,but there are also shortcomings,such as inaccurate hand models and low tracking rates,which have not been applied in practical scenarios.Therefore,this article designs a human hand detection and tracking system for collaborative robots,and conducts research on human hand detection and tracking,and applies it to practical scenarios of collaborative robots.The following work has been done.(1)A hand detection algorithm based on the combination of skin color and hand model is proposed to address the issue of factors such as skin color changes and background interference of rectangular boxes that may affect the accuracy and stability of detection.Firstly,a twodimensional Gaussian statistical distribution of the skin in the YCbCr color space is used to preliminarily detect the human hand,and then DoG is used to detect the palm and elliptical approximate fingers to determine the contour of the human hand.This method can successfully detect and track human hands even under scale changes,non rigid transformations,and rotations.By training,the parameters of the skin dataset are obtained without manually adjusting the threshold,with high accuracy.It can accurately extract human hands in both simple and complex scenes,and also avoids interference caused by rectangular frames surrounding other backgrounds in the image for subsequent training.(2)A particle filter human hand tracking algorithm based on the two-dimensional Gaussian distribution of YCbCr color space is proposed to address the problem of nonlinear non Gaussian model tracking of human hands.Firstly,it is necessary to model the color of the human hand skin area,obtain the color feature information of the target,and calculate the mean and variance of the Gaussian distribution of skin color in the YCbCr color space.Then initialize the particles,determine the particle motion model,and use the state of the particles and the motion model of the target to predict the state of each particle and obtain the state at the next moment.Assign particle weights based on the similarity between the pixel positions of each particle in the current frame and the skin color model,normalize the weights of all particles,and finally resample to determine the position of the updated human hand.(3)In object detection algorithms based on deep learning,the One stage object detection algorithm only requires one forward calculation,does not need to generate candidate regions,and does not require the operation of the RoI Pooling layer.The computational complexity is relatively small,the speed is faster,and real-time application scenarios can be used.Therefore,the YOLOv5 deep learning object detection method in the One stage object detection algorithm was used to train the calibrated dataset,and an accuracy of 95.5% was achieved.(4)The above methods of human hand detection,human hand tracking,and object detection have been successfully applied to the UR3 collaborative robot,achieving the robotic arm to grasp objects,track and place them in the human hand,and complete the recognition and tracking system,providing a method for the human-machine interaction landing of the collaborative robot.