Gesture Recognition In Augmented Reality Based On Deep Learning

With the increasing demand for human-computer interaction experience and the development of computer technology,gesture interaction has become a new generation of popular human-computer interaction methods.Gesture recognition is a key component of gesture interaction technology.Gesture recognition can recognize user-specific gesture images to parse out the user’s command information to the computer,which can significantly improve user experience and interaction efficiency.In emerging interaction scenarios such as augmented reality,vision-based gesture recognition technology is a popular research direction.The recognition accuracy of gesture recognition has already reached a high level.Most of the existing vision-based gesture recognition methods study gesture image recognition under ideal input,but in augmented reality interaction scenarios,due to the complex and changing interaction scenarios,limited camera capture range,frequent iterations of interaction gestures,user handicap or handheld objects,etc.,the gesture recognition under augmented reality is easily affected by"weak information conditions "The existing gesture recognition algorithms are facing corresponding problems,making the recognition accuracy decrease,which leads to limited gesture design,poor user experience,unfriendly to people with hand disabilities,and even serious safety problems.Gesture recognition in complex scenes,gesture recognition in the presence of local occlusions,and gesture images with limited training data all face these problems.An improved gesture recognition method is proposed to address these problems arising in augmented reality.There are three main research aspects as follows.(1)To address the problem that it is difficult to extract the overall semantic information of gesture images effectively in complex scenes,a gesture recognition algorithm based on multimodal feature fusion is proposed.Semantic segmentation and hand key point recognition are introduced to extract gesture image features,and gestures are recognized based on attention mechanism fusion of the above multimodal features to improve the accuracy of gesture recognition in complex scenes.(2)To address the problem that information in key regions is lost when there is partial occlusion in gesture images,a partial occlusion gesture recognition algorithm based on feature recovery is proposed.A feature recovery network is constructed,and consistent regularization constraints are added to different locations before the network is trained,which improve the accuracy and robustness of gesture recognition in partially occluded scenes.(3)To address the problem that collecting data is costly while the application scenarios are various,a limited-sample gesture recognition algorithm based on transfer learning is proposed.The ideas of transfer learning and co-training are introduced to make the large-sample model and the small-sample model guide each other in training,specialize the feature extraction ability of the large-sample model and strengthen the feature extraction ability of the small-sample model,so as to improve the accuracy of the model trained on the small-sample data set.