Research On Algorithms Of Hand Gesture Recognition Based On Multi-sensor Fusion

With the development of perceptual computing and sensor integration technology,more and more hand gesture motion data can be captured in real time by various sensor devices,which provides a new data-driven way for human-computer interaction.Deeply analyzing hand gesture data based on wearable devices and fully mining the hidden information of the data can enhance the interactivity of hand gesture recognition,and widely used in smart home,telemedicine,virtual reality and other fields.Therefore,this thesis is based on multi-sensor information fusion technology,with hand gesture recognition as research goal,combined with deep learning algorithms,to improve the effect of hand gesture recognition from data feature level and spatial distribution of sensor level.The main work of this thesis is as follows:(1)Hand gesture recognition based on deep feature fusion network.First,this thesis designs a data glove integrated with double arm rings to collect hand posture and subtle movement information.Then,hand gesture recognition based on deep feature fusion is proposed on the basis of analyzing the structure of hand gesture data.From the perspective of data feature,according to the idea of complementary feature information,DFFN uses feature fusion to analyze and recognize a variety of hand gesture actions.In addition,in the process of model training,residual module is introduced to prevent the disappearance of gradient.Finally,the feasibility and effectiveness of DFFN are verified on multiple data sets.(2)Hand gesture recognition based on weighted feature enhancement network.From the aspect of data features,hand gesture recognition based on weighted feature enhancement is proposed according to local interaction characteristics of hand gesture data.Firstly,mutual information is used to quantify the correlation between different sensor features and classes,and a weight calculation model is constructed to calculate the correlation degree.Then,WFEN is introduced to extract interactive information of adjacent joint points,which characterizes local interactivity of hand gesture data.Finally,combined with LSTM to solve the temporality and long-distance dependence of dynamic hand gesture data,the accuracy of hand gesture recognition is improved.In addition,in the process of model training,based on softmax loss function,Fisher linear criterion is used to construct the loss function to optimize the recognition model.(3)Hand gesture recognition based on spatial-temporal graph neural network.From the perspective of spatial distribution of sensors,hand gesture recognition based on spatial-temporal graph neural network is proposed to represent the spatial correlation between hand gesture data on the basis of spatial location information of sensors.Firstly,spatial location characteristics of wearable sensors are described by graph structure,and GCN is used to aggregate spatial correlation information of hand gesture data.Then,according to temporal information,spatial features of different samples from the same joint point are transformed into temporal feature series,and GRU is introduced to solve temporality and long-distance dependence of dynamic hand gesture,so as to enhance the performance of hand gesture recognition.(4)Design and implementation of hand gesture recognition integrated system.Based on object-oriented design idea,combined with the research results of this thesis,hand gesture recognition integrated system is designed and implemented in layers and modules.The system visualizes the key steps of hand gesture recognition,conducts experimental verification and results analysis on the research results of this thesis,and provides experimental basis for further large-scale application and expansion.