| Gestures are a natural and fundamental tool for human interaction and have been used to convey information since long before the advent of language.Through a series of hand movements and finger positions,gestures can convey information and allow complex operations to be performed with ease.As such,gestures can be used as a highly adaptive form of interaction in human-computer interaction,simplifying interaction by eliminating physical contact between the device and its user.Gesture interaction can provide more intuitive interaction and richer interaction effects,which can better meet people’s needs and expectations for interaction methods.In real life,people often need to recognize gestures to interact in a variety of complex environments,which requires gesture recognition methods that are highly resistant to complex backgrounds and lighting while maintaining recognition accuracy.Traditional methods based on machine vision or manually designed gesture features are mostly based on simple backgrounds,or ignore the effects of backgrounds.The accuracy and robustness of gesture recognition in complex backgrounds are not ideal and are not conducive to practical production applications.In this thesis,the problem of single gesture and multiple gesture recognition in complex backgrounds is investigated and the relevant software system is designed and developed for practical application of gesture recognition.The main elements are as follows:(1)To solve the problem of low accuracy and slow recognition speed of single gesture recognition in complex backgrounds due to the interference of single gesture images,a single gesture recognition algorithm in complex backgrounds based on structural reparameterization and attention mechanism is proposed.By using the structural reparameterization method and applying it to the residual structure,the redundant branching structure is removed at the deployment stage to improve the recognition speed of the algorithm;at the same time,an attention mechanism module is introduced to focus the algorithm on gesture features by using its feature of weighting different channel features to reduce complex background interference;finally,two data enhancement methods,cutout and affine transformation,are used to train the algorithm to suppress Finally,the algorithm is trained using two data enhancement methods,cutout and affine transform,to suppress complex background noise input and enhance the data,reducing overfitting while improving the robustness of the algorithm.In a comparison experiment on a complex background gesture dataset,the recognition accuracy reached99.9% and the recognition speed reached 200 FPS,proving the effectiveness of the proposed algorithm.(2)To solve the multi-gesture recognition problem in complex backgrounds,an improved YOLOV5-based multi-gesture recognition algorithm in complex backgrounds is proposed.The algorithm uses the YOLOV5 target detection network as the basic network in order to detect each gesture position separately and recognise multiple gestures.By replacing the CSP1＿x module in the YOLOv5 backbone network with an efficient layer aggregation network to obtain a richer combination of gradient paths,the network learning and expression capabilities are improved and the network recognition speed is enhanced.The CBAM attention mechanism module is introduced to achieve filtering of gesture features in channel and spatial dimensions to reduce various types of interference in complex background gesture images and to enhance the robustness of the network against complex backgrounds.Validation experiments were conducted on the complex background gesture datasets Ego Hands,Tiny HGR and the self-built dataset,with recognition accuracies of 75.6%,66.8% and 83.2% for m AP@0.5:0.95 respectively,and recognition speed of 64 FPS for 640×640 size input images.The experimental results demonstrate that the proposed method can recognize gestures quickly and accurately in complex backgrounds,and achieve better recognition accuracy and robustness than comparative algorithms such as YOLOv5 l and YOLOv7.(3)Combining the characteristics of gesture images and gesture recognition processes in complex backgrounds,the functional requirements of gesture recognition systems in complex backgrounds are analyzed,and the overall design and functional implementation of gesture recognition systems in complex backgrounds are carried out.The software is capable of implementing a complete set of gesture recognition processes in complex backgrounds,from data acquisition,data management,data annotation,model training and gesture recognition,providing a reference for the practical application of gesture recognition in complex backgrounds. |
