Hand Gesture Recognition Based On Ultrasound Of Smartphone And Deep Learning

Hand gesture recognition technology has attracted great attention of researchers in the field of ubiquitous computing.It can not only provide us with a variety of convenient and simple control gestures,but also bring a human-computer interaction mode more in line with daily life habits.Furthermore,smartphones have been widely used,and the performance of its built-in sensors has also been greatly improved.As a result,it has become a hot research topic to recognize hand gestures by using the built-in speakers and microphones of smartphones.This dissertation mainly focuses on the implementation of hand gesture recognition based on ultrasound by using the smartphone's built-in speaker and microphone.This scheme utilizes the speaker to transmit ultrasound and the microphone to capture the changed signal caused by hand movement.By analyzing the received signal,eight hand gestures can be classified,including towards and away from the smartphone,English letters,clockwise.This paper tests the designed system in a noisy environment and quiet environment,and the experimental results show that our system can identity gestures with an accuracy up to 99%.The main contents and innovations of this paper are as follows:(1)This paper summarizes the current hand gesture recognition technique and studies the technique based on ultrasonic signal of smartphone.This paper also introduces the existing hand gesture recognition schemes based on ultrasound of smartphone,summarizes the involved algorithms and related knowledge,and illustrates the algorithms used in our designed system.(2)Based on the existing hand gesture recognition schemes using ultrasound of smartphone,a hand gesture recognition system based on ultrasonic signal of smartphone is designed.This system turns the smartphone into an active sonar and uses the Doppler effect to recognize hand gestures.(3)A method of hand gesture recognition by using the convolutional neural network and a gated recurrent unit network is proposed.This paper builds a convolutional neural network and a gated recurrent unit network to classify hand gestures,and evaluates these two networks in two environments.The experimental results show that these two networks can achieve a recognition accuracy above 99%and 98%,respectively.