Research On 3D Hand Key Point Detection Based On Depth Image

Our era is running on the information superhighway.Technology advances with-out boundaries,and machines are getting smarter.Meanwhile,people's application needs for some fields like human-computer interaction,robotics services,virtual somatosen-sory and so on are also increasing,and the requirements are also more detailed and pre-cise.Since human hands can transmit a large amount of rich information,it is very im-portant to study the ability of machines to detect hands' joint in practical applications.In the early days,people used sensors to implement hand detection tasks in some scenes,but this method is not convenient and costly.Since deep learning has shown great research potential,using deep learning to detect hands' joint has become a mainstream research direction.Combining the needs of actual application scenarios,this paper makes an intensive study on detecting 3D position of hands' key points based on depth images.And designs two methods for the problem of high self-similarity of hand joints and real-time detection.The main research work is as follows:(1)A 3D hand key point detection method integrating attention mechanism based on densely set anchor points on the depth image is designed.Aiming at the problem of high self-similarity of hand joint points and the problem that depth images contain a lot of information but the processing is complex,this paper explores extracting hand images from depth images by thresholding.And based on the method of using anchor points to return the three-dimensional coordinate of key points on the depth image which can be di-vided into three branch tasks: plane offset estimation,depth value estimation,and anchor point suggestion,the attention mechanism is introduced in this paper.The top-down and bottom-up feature learning is integrated into the weight distribution link of anchor points that may contain key points,so that the model pays more attention to the information around the anchor points in the deep feature map to assign weights more accurately.This method is trained and tested on the NYU dataset,and use random erasing for data en-hancement.And the experimental result use average 3D distance error performance in-dex to compare with the advanced methods.Compared with the benchmark method,the method in this paper has a 0.06 mm accuracy improvement.(2)A lightweight 3D hand key point detection method based on the depth image is designed,which takes into account accuracy and speed.In view of the application in actual scenes that need to maintain accuracy and ensure lightweight and real-time operation,this paper introduces the mainstream lightweight network Mobile Net V2 and Shuffle Net V2 structures into the 3D hand key point detection.Use them as the backbone network reduc-ing the training parameters to compress the network model.The backbone network will learn the underlying features,divide the features into two stages and put them into three branches for key point position regression.This method uses 4 performance indicators to experiment on the NYU and ICVL dataset and compare with the advanced methods.This paper makes a self-comparison of whether the two structures are integrated with the at-tention mechanism,and also compares the compression effects of the two lightweight structures.The fusion attention mechanism based on Shuffle Net V2 designed in this paper compared with the method based on Mobile Net V2,the accuracy and speed of the former method are better.The model size is 89.1MB,which is 47.6% compressed compared to the benchmark model.And the model's FPS value is 126.3f/s so as the speed increased by 20.2%.