Research On Handwriting Recognition Based On Passive Audi

Smart devices in the Internet of Things era are becoming lighter and smaller,which brings challenges to human-computer interaction,especially text input.As a natural and convenient way of interaction,handwriting recognition is gradually becoming an effective way to solve this problem.Handwriting recognition solutions in existing research mainly utilize computer vision technology,wearable sensors,etc.The main disadvantage of the computer vision-based solution is that it is difficult to guarantee the security of the user’s personal privacy data,and it is greatly affected by light,background,and shooting angle.Capturing hand motion information based on wearable sensors can achieve good accuracy,but requires users to wear additional sensor devices.In recent years,the use of acoustic signals for handwriting recognition has gradually become a hot topic due to the advantages of acoustic signals for privacy protection and no need for additional equipment.This paper proposes a handwriting recognition system based on passive audio,which uses the built-in microphone of the device to capture the audio signal generated by the finger sliding on the object plane to recognize the content of the handwriting input.The main contributions of this paper are as follows:(1)A handwriting input segmentation algorithm based on signal energy is designed and proposed,which can accurately segment effective signal segments from collected handwriting signals.The fluctuation range of the normalized signal frame energy becomes smaller,so that the gap between the effective signal and the noise signal in the audio is enlarged,and the interference of environmental noise can be effectively suppressed.The algorithm takes the stroke segment as the smallest segmentation unit,detects the start point and end point of each handwritten stroke segment in the signal according to the signal frame energy,and concatenates them into character signals according to the time difference between the handwritten strokes.(2)The acquisition process of audio data is cumbersome and time-consuming.Adversarial generation networks are introduced to expand the data.In the case that only a small amount of real audio data needs to be collected,a large amount of realistic synthetic data is generated by the generator in the generative network,which greatly saves manpower and time costs.(3)In order to make the classification model more lightweight,the lightweight classification network Shuffle Net V2 is adopted,and its structure is adjusted.The model has achieved a good balance in recognition accuracy and computational complexity,which is very friendly to devices with limited computing and storage resources..(4)In order to improve the efficiency and accuracy of handwritten text input,a candidate word algorithm is designed based on the N-gram language model to automatically provide users with the highest matching candidate word according to the prefix of the input word when inputting text.In this paper,the model is deployed on an Android smartphone,and the classification performance is tested from the dimensions of different plane materials,different environmental noises,and the number of candidate words.The test results show that,without the support of additional equipment,the model has an average recognition accuracy of 92.2% for handwritten numbers and uppercase letters,and an average recognition accuracy of 94.3% for words given five candidate words.