| Gesture recognition has the benefits which are natural,intuitive and convenient to operate,and is extensively utilized in the domain of human-computer interaction.In terms of the technical distinctions in acquiring gesture input data,gesture recognition can be compartmentalized into two methods: wearable device-based and vision-based.In the visionbased gesture recognition method,static RGB(Red Green Blue)gestures can be captured anytime and anywhere with ordinary cameras,without the need for additional cameras to collect depth information,which has higher practical value.Static RGB gestures have the characteristics of diversity,ambiguity and high degree of freedom.In addition,factors such as lighting conditions and background complexity have a greater impact on the accuracy of recognition.The deep object detection networks have excellent performance for gesture detection tasks under complex conditions.However,most object detection models are tricky to be embedded in small terminals due to a huge amount of computation and parameters,which limits the practical application scenarios of the models.Therefore,this research constructs a lightweight gesture recognition model and transplants the model to use on small devices.The specific research work is as follows:(1)Construction and preprocessing of the gesture dataset.The gesture dataset of this study is collected in two ways,online and offline.The dataset contains 15 gesture images of different scales as well as backgrounds,and each gesture has a certain social semantics.The data collection environment basically covers the gesture detection environment in general scenarios,meets the actual detection requirements,and uses the annotation tool for annotation.In order to enlarge the data sample size and prevent the overfitting of the model,data enhancement is carried out by means of saturation adjustment,color change,Mosaic,etc.(2)Research on lightweight gesture detection model.This research builds an efficient and lightweight gesture detection model based on the YOLOv4-tiny network structure.First of all,this paper introduces ghost feature mapping with stronger representation and lower cost.The feature extraction module of the network is optimized to enhance the ability of the network to obtain multi-scale gesture features.At the same time,the computational complexity of the network is reduced.Then,a variety of activation functions are compared and analyzed,so Swish is used as the main activation function to improve the network’s ability to non-linearly model gesture data,and further improve the accuracy of gesture recognition.Finally,the embedded squeeze excitation module is used as the channel attention mechanism of the network,which integrates the global feature information,and recalibrates it according to the feature weight,reduces the interference of irrelevant information such as background,and improves the network’s ability to express detailed features.The experimental results on the self-built gesture dataset show that,compared with other comparative models,the model proposed in this paper can accurately classify and detect multi-scale gestures under different environmental conditions in real time,with the highest recognition accuracy and better performance.The detection speed is especially advantageous for small-scale gesture recognition.Meanwhile,the recognition accuracy of the model proposed in this research has as well as been improved on public gesture datasets,indicating its robustness and generalizability.(3)Development of interactive gesture recognition software.In order to expand the application scenarios of gesture recognition,the gesture detection model was transplanted to the mobile phone,and the gesture recognition software was designed and developed.Based on the gesture images captured by the camera in real time,the software is able to infer the model in an offline environment,and implement three interactive modules: gesture detection,gesture photography and gesture animation.The software is equipped with this model and realizes the interactive gesture recognition function,which improves the practicability of this model. |
