Research And System Design Of Two-hand Gesture Recognition Method Based On RGB-D

The RGB-D-based two-handed gesture recognition method is an emerging contactless human-computer interaction technology with high application value in many fields such as smart cars,smart homes,VR/AR,etc.However,it is still challenging to establish a stable,accurate,and efficient two-hand gesture recognition system based on RGB-D information.Left and right-handed region detection and hand pose are essential research in two-handed gesture recognition tasks.The main research goal of this paper is to achieve accurate and fast left and right-handed detection through RGB-D images.At the same time,taking advantage of the fact that depth information is not affected by illumination and background environment,this paper achieves accurate and robust two-hand gesture keypoint estimation and two-handed gesture recognition through single-frame image processing and convolutional neural network methods.The main research works and contributions of this paper are:(1)Implementation of left and right hand detection algorithmYOLO v4 does not have high accuracy in detecting small targets,it will miss detection when the proportion of gestures in the image is small,and the detection results are inaccurate due to the large proportion of background weights.In order to solve these problems,this paper proposes an optimization scheme by analyzing the performance of YOLO v4.This paper proposes a high-speed left and right-hand detection algorithm based on improved YOLO v4(2)Implementation of depth information-based hand pose estimation and twohanded gesture recognition algorithmThis paper introduces depth information for gesture key point estimation of left and right-hand regions and improves regression accuracy by adding distance information.A two-stage gesture keypoint estimation algorithm is proposed to solve background and human body influence on gesture regression accuracy.Finally,template matching is performed for the predicted vital points.The two-handed gesture recognition results are obtained by analyzing the left and right-hand bending states.(3)Two-handed gesture recognition systemThis paper builds a two-handed gesture recognition system based on RGB-D.Using Py Qt5 and its toolbox,a friendly and straightforward two-handed gesture recognition system GUI is created,providing left and right-hand detection,gesture pose estimation,and two-handed gesture recognition functions.