Design And Implementation Of Hand Pose Estimation System Based On Deep Learning

The rise of deep learning has brought AI into people's daily life.Digital assistants,like Siri from Apple and Cortana from Microsoft,have shown bright prospects.For most people,almost all daily activities,like driving,writing,taking phone calls,involve the use of hands.Interpreting hands can help digital assistants get more comprehensive information on scenes of people's daily life,and so offer better services to users.On other hand,hand pose estimation is the key part of HCI(Human-Computer Interaction)that merely involves hands.The goal of hand pose estimation is: from a RGB image that contains a hand,estimating the relative coordinates of hand key points in three-dimensional space.This thesis designs and implements the pipeline of hand pose estimation,which consists of the following subsystems:(1)“hand detection” subsystem,which from a RGB image that contains a hand detects and then crops out the hand,and is implemented using the SSD object detection technique;(2)“hand key points estimation” subsystem,which from the cropped out image estimates the locations of the hand key points in the image.;(3)“hand pose estimation” subsystem,which from the locations of the hand key points in the image estimates the relative coordinates of those key points in three-dimensional space.This thesis also designs and implements a fingerspelling recognition system to explorer the applications of hand pose estimation system as a hand features extractor.The innovations of this thesis are the following:(1)Considering the similarities between hand key points estimation and 2D human pose estimation,this thesis bases the CNN structure of “hand key points estimation” subsystem on CPM(Convolution Pose Machine);(2)This thesis views a hand pose as rotating from a canonical hand pose,and decomposes this subsystem into “canonical hand pose estimation” subsystem and “view angle estimation” subsystem,and so makes training easier.