The Research On Fine-grained Gesture Input Technology Based On Acoustic Sensing

Recently,wearable devices have become increasingly popular in our lives because of their neat features and stylish appearance.A growing number of people use them to make a phone call,send messages and even play games.However,their tiny sizes bring about new challenges to human-device interaction especially in texts input.Although some novel methods have been put forward to deal with this problem,they have different defects such as internet required,vulnerable to ambient noise,leaking privacy,or requiring users to attach specialized hardware with them.As a result,we propose an acoustic-based fine-grained finger gesture input technique,by which finger gestures can be recognized and texts can be entered with a finger writing in the air without wearing any additional devices.This method breaks the limitation of traditional interaction such as the virtual keyboard on small screen,eliminates interfere caused by screen size,user diversity,noise and weak internet access.In this circumstance,any smart device can use this technique as long as they equip with a pair of microphone and speaker.Differing in many previous works,this technique utilizes the physical acoustic principle to model the finger movement,and extracts the intrinsic property of each finger gesture by signal processing and image processing.Therefore,it can work in a training-free style which reduces the training overhead and improves system scalability.In addition,we design a robust continuous signal segmentation algorithm,by which system can detect the start point and end point of finger gesture signal precisely even in disturbed environment.Moreover,a Bayesianbased linguistic model is proposed to infer,correct and predict words automatically.In this way,user can input text more efficiently.We implement a real-time system based on this technique,named EchoWrite,with a commercial smart phone and a smart watch,and conduct comprehensive experiment to evaluate its performance.Experimental results show that EchoWrite can achieve finger gestures recognition with accuracy of 94.7%.The average response time of recognition is about 176 ms.With three candidates,word recognition accuracy is up to 94.9%.In general,the correction algorithm can improve word average recognition accuracy by 4.4%,enables users to enter texts at a speed of 7.5 words per minute.Moreover,we conduct a NASA-TLX task load index evaluation,and result illustrates that EchoWrite provides favorable user experience of humandevice interaction.