Research On 3D Gesture Recognition And Application Based On RealSense Depth Camera

With the continuous improvement of depth camera and gesture recognition technology,the gesture recognition technology based on depth camera is widely used in various human-computer interaction scenarios,such as Virtual Reality(VR),Games,Intelligent Robotic Arm,etc.Although gesture recognition models can better eliminate issues such as background illumination and color information sensitivity using depth information obtained from depth cameras,there are still problems such as environmental foreign object interference,relatively low accuracy,and insufficient use of depth information,as well as high latency when applied to web page operations.In order to solve the above critical problems,Based on the Intel Real Sense D415 depth camera,thesis proposes a 3D dynamic gesture recognition network model(3D-DGR)that combines RGB image and depth image information,and applies the model to web page operations to successfully realize natural human-computer interaction with gesture control corresponding to web page operations.The main research contents of thesis are as follows:1.To address the problem of image preprocessing operations,thesis proposes a method to segment gestures on RGB and depth images simultaneously using the depth information obtained from the D415 depth camera.First,align the RGB image with the depth image,and then set a certain depth distance threshold to complete the simultaneous segmentation of the gestures in the RGB and depth images,achieving the effect of reducing background interference.2.Aiming at the issue of relatively low accuracy and poor real-time performance in the application of gesture recognition models in web page operations,thesis proposes a new 3D dynamic gesture recognition network model,3D-DGR,which combines RGB image and depth image information.Based on the self built dynamic gesture dataset,a series of comparative experiments have shown that the accuracy rate of this method is as high as 98.60%,1.93% higher than the dynamic gesture recognition model using only RGB images,and the performance is better than backone’s model using Alex Net and VGG-16.3.Using the depth camera and 3D-DGR to conduct relevant research on web pages through Chrome Driver,9 operations such as web page up,down,zoom in and zoom out were successfully achieved,and the response time was all less than 0.73 seconds.Through the gesture recognition operation web page experiment,it can be proved that the gesture recognition operation model proposed in thesis can meet the real-time and other task requirements.In summary,thesis proposes a model with higher accuracy and better robustness for the key problems of applying gesture recognition model based on depth camera,and applies it to web page operation with smooth user experience.