Real-time Character Input System Based On Dynamic Gesture Recognition

With the widespread application of computer technology in the scientific field,the emergence of Human-Computer Interaction(HCI)has had a positive impact on the use of computers.In order to make HCI seem more natural and intuitive,people introduce Hand Gesture Recognition(HGR)technology into the field of human-computer interaction.Gesture recognition and gesture-based interaction have attracted more and more attention in HCI.In this research,a real-time character input system based on dynamic gesture recognition was constructed,which mainly adopt deep learning technology to realize dynamic gesture recognition,and the keyboard was controlled by gestures based on Lab VIEW platform to realize the function of real-time typing.First of all,the Jester database containing 27 gesture categories were applied to construction of the gesture recognition algorithm.It mainly includes the long sequence modeling in the dynamic gesture video based on the Temporal Segment Network(TSN)model,the extraction of gesture information with the help of sparse sampling scheme,and the integration of gesture motion information under the average fusion method.Then,the classification results were obtained by softmax layer.Secondly,a residual neural network(Residual neural network,Res Net)was used as the basic network model of TSN in order to improve the performance of the network.Finally,the new fusion function of Temporal Relational Network(TRN)was used to replace the average fusion of TSN.The result proved that the addition of TRN makes the gesture classification effect better.The training accuracy rate of this study on the Jester data set was 94.319%,and the test accuracy rate was 94.30%.At the same time,the monocular camera was used to collect gesture video streams for real-time recognition of dynamic gestures,and the gesture recognition rate was basically maintained between 90% and 99%.In this article,a real-time character input system based on dynamic gestures was designed,which could predict dynamic gestures in real time and control the keyboard through gestures to realize the real-time input of characters such as Chinese pinyin,numbers,and English letters,as well as the up and down page flipping function of Microsoft Office Power Point(PPT).The system not only reduces the complexity of extracting features in traditional gesture recognition methods,but also has a import significance in practical applications.For the elderly,people with poor eyesight and users who need teaching demonstrations,the gesture typing input system is more convenient and intuitive.