Research On Gesture Motion Trail Recognition Based On Feature Fusion Network

On account of the predominance in human labour and life,hand gesture,especially its motion trail,is an ideal vehicle of next generation human-computer interface owing to abundant nonverbal information,and has promising application prospect in the field of intelligent household,automatic driving and augmented reality.Nonetheless,current gesture motion trail recognition(GMTR)algorithms lack either corresponding trail capture method or strong robustness to varied application scenarios.Thus,this thesis decomposes GMTR problem into three sub-missions,namely gesture detection,gesture tracking and trail recognition,and based on deep learning theory and gesture's particular characters,proposes an effective GMTR algorithm by utilizing feature fusion module to optimize its three sub-missions.From overall perspective,the main contributions of this thesis are as follows:1.This thesis proposes a static gesture detection method based on multi-scale feature map fusion.To meet the challenge of scale variation of gesture in application,this thesis proposes a dilated convolution block composed of convolution branch with different dilation rate to select receptive field adaptively,and a feature fusion block to merge feature maps with different scales.On this basis,this thesis designs a top-down feature fusion structure,which can facilitate the fusion of semantic and positional information,and accordingly enhances gesture detection performance.2.This thesis proposes a gesture tracking algorithm based on attention mechanism.On the basis of gesture detection results,this thesis generates gesture attention map through hand key-points detection model,and superimpose it on feature maps of gesture match network with siamese structure,to improve network performance by reinforcing motion information,and implements gesture tracking through gesture matching in consecutive frames utilizing siamese network.3.This thesis proposes a trail recognition method based on context-dependent information.This thesis gains gesture trail by recording the center of its tracking bounding box,and designs a light-weighting neural network to tackle single trail recognition considering its particular characters.To impair the adverse impact of incomplete and invalid trail,this thesis processes trails integrally as a sequence,utilizes the lightweighting network as their feature information decoder,and exploits their context-dependent information with connectionist temporal classification module,to improve overall recognition accuracy.Meanwhile,this thesis builds a gesture-targeted data set to train convolutional neural networks used and provide a platform for performance comparison.The experimental results demonstrate that the proposed algorithm achieves excellent performance and efficiency,and can address the GMTR problem effectively in natural scene.