Research And Implementation Of Keyboard Input Monitoring Method Based On Audio Characteristics

With the continuous development of science and technology,in our social life,the way of using keyboard to access the Internet for communication and using keyboard to input key information for computing occupies more and more great proportion.Using a large number of acoustic signals generated by tapping keyboard may give eavesdroppers the opportunity to peep remotely,and then by carrying out corresponding mathematical analysis can make it possible for eavesdroppers to restore some sensitive information generated by tapping keyboard.With the increasing emphasis on security and privacy issues,the topic that by means of keystroke monitoring can directly extract the user's sensitive information can from the acoustic signals generated by keystroke has attracted extensive researcher's attention.The existing keyboard monitoring schemes: the first schemes is about analyzing the audio of every keystroke on the keyboard,establishing a continuous learning process,and obtaining a training library based on keyboard.Although machine learning can successfully identify every keystroke,in the attack scenario,we have no chance to carry out correspond machine learning on the keyboard of the attacked person at the same time.The huge cost of learning at the same time is also one of the drawbacks of this scheme.Other scheme is based on location fixity.Even though the relative position of the smartphone used for monitoring and the keyboard being monitored is fixed,it is unrealistic to rely on the assumption that the relative position of the keyboard is known in advance in the actual monitoring scenario.In order to solve this problem,this paper proposes a method to monitor keyboard keys without fixing mobile devices and keyboards in advance.It uses two microphones of a single mobile phone to record the audio of keyboard keys.By utilizing the dependence between voices and the specific layout of keyboard,a geometric method is developed to estimate the relative position and angle between mobile phones and keyboards,and then the method is used to estimate the relative position and The relative position between the mobile phone and the keyboard calculated by the method above,and the time difference between the calculated keyboard audio and the two microphones,can effectively monitor the keyboard keys.In addition,we propose a method based on the above process,which can effectively improve the recognition accuracy of keyboard keys by analyzing the Meier cepstrum coefficients of a single key and K-means clustering method.In the paper,we use Samsung smart phone to simulate the experiment.The results show that the positioning scheme achieves great high accuracy.Compared with the existing keyboard monitoring methods,the keyboard recognition accuracy is improved by 32.6%.