Research On User Inputs Recognition Based On Mobile Phone Inertial Sensor

With the popularity of smart phone and the rapid development of mobile applications in recent years,smart phone plays an increasingly important role in people's daily life.Meanwhile,the security of smart phone has attracted more and more attention and concern.Inputs recognition based on smart phones is a hot topic in the field of data security and privacy protection,it uses all kinds of sensors of mobile phones as side channels to recognize and even steal the input information of the user's virtual keyboard.At present,the existing researches have low recognition accuracy,and mainly focus on the problem of text recognition,ignoring the recognition of passwords which is more valuable than text.This paper aims at the above problems,uses phone's inertial sensor as a side channel to study and analyze the recognition problem of user inputs on the virtual keyboard.The main research works of this paper are as follows(1)Based on the analysis of smart phone inertial sensor and virtual keyboard input behavior,this paper studies and designs an inputs recognition scheme by using accelerometer and gyroscope.The scheme generates the attitude angle information by integrating accelerometer data and gyroscope data at first,and extracts the classification features for recognizing the user inputs from the attitude angle,then several types of classifiers are trained,it uses the combination of voting strategy to build an ensemble classifier,the pre-trained classifier is used to classify the attitude features in the recognition stage,which achieves the purpose of inferring user inputs.(2)This paper designs and implements an attitude estimation algorithm based on complementary filtering and the click detection algorithm based on statistics.The former utilizes the complementary characteristics of the accelerometer and the gyroscope in the frequency domain,it uses the complementary filter for data fusion,reduces the computational complexity while enhancing the accuracy of the attitude measurement.The latter collects the acceleration data when the user clicks the screen and counts the features distribution range during the training phase,it detects the occurrence of a click event by verifying whether the acceleration eigenvalue falls within the statistical range in recognition phase.(3)This paper designs and implements user inputs recognition experiments.The experimental result shows that the recognition accuracy of individual characters can be improved greatly by increasing the features and using a combination classifier.At the same time,this paper builds the hidden Markov model,and uses accelerometer data as the observation sequence,which can greatly reduce the search space of password,thus saving the time needed to break the password.For the purpose of security,we propose some defense methods and a general security framework to defend against more kinds of attacks that sniffing user privacy through sensors at last.