Research On Pervasive Privacy And Security Techniques Leveraging Acoustic Sensing

With the development of mobile communication technologies and highly-integrated electronics,the mobile devices become ubiquitous in human daily life and work.Different from typical computing devices(e.g.,desktops,servers,etc.),mobile devices could be carried by users to any places and continuously provide efficient and reliable services(e.g.,news reports,data storage,localization services,etc.),because of its portability and wireless connections.However,the privacy and security problem for mobile scenarios are also introduced with the strong connections by mobile computing.The mobile privacy and security problem can be separated into mobile attack and mobile defense.Existing studies reveal potential attacks of mobile computing to arouse users’ awareness for their privacy and security,as well as inspire service providers to develop corresponding solutions to safety guard,such as reveling infrastructure attack,virus/malware,botnet,side-channel attack,etc.On the other hand,despite revealing potential attacks,more straightforward approaches to saving user privacy is to design mobile security defense solutions,such as designing intrusion detection,trust platform construction,two-factor authentication,user authentication,etc.Existing researches on these two fields concentrate on investigating the privacy and security problems of mobile devices in cyber spaces,while outlooking the pervasive privacy and security problems for mobile devices under the strong connection between physical and cyber spaces in mobile scenarios.This thesis aims to investigate the pervasive privacy and security problems under the strong connection between physical and cyber spaces in mobile scenarios,including designing user authentication systems and revealing side-channel attacks.This thesis employs the acoustic sensing techniques realized by the audio devices widely-integrated into mobile devices(including speakers and microphones)to explore and study the pervasive privacy and security problems in mobile scenarios,which not only investigates the impact of physical space’s environmental constraints on applying acoustic sensing in pervasive privacy and security problems,but also implementing acoustic sensing-based mobile pervasive privacy and security solutions for cyber spaces.The research on the mobile pervasive privacy and security leveraging acoustic sensing faces many technical challenges.(1)The integrated audio devices of commercial mobile devices have limited capability due to their limited cost,which leads to the fact that basic acoustic sensing techniques cannot satisfy the fine-grained sensing requirements in pervasive privacy and security problems.(2)The acoustic sensing by the integrated audio devices in mobile devices is omni-directional,which indicates that the acoustic signals emitted from the speaker carrying the individual privacy propagate through multiple paths and probably leak to adversaries finally.(3)The acoustic signals are widely existed in physical spaces(e.g.,mechanical noises,human speech voices,etc.),and easily mixed into the target acoustic signals so as to significantly interfere the acoustic-based feature extraction or recognition performance.(4)Due to the size-limited electronics and power-limited battery,mobile devices can only provide limited computing capability,and thus requires the application adaptation for the devices.Hence,the solutions for mobile devices are required to be lightweight and computational efficient.To address these challenges,this thesis mainly investigates three problems as follows:First,this thesis proposes a lip reading-based user authentication system to provide mobile defense for pervasive privacy and security,which extracts the unique behavioral characteristics of mouth movements during speaking leveraging acoustic sensing for user authentication.This work observes during a speaking behavior of humans,not only the speech voice can be used to distinguish different individuals,but also the mouth movements could exhibit the individual uniqueness.Inspired by this observation,this work employs Doppler-based acoustic sensing techniques and deep learning approaches to explore such a uniqueness and realize the user authentication for mobile devices.We first investigates the Doppler shift of acoustic signals induced by mouth movements during speaking,and find that different individuals have distinct patterns of mouth movements.To characterize the mouth movements,we propose a deep learning-based approach to extract efficient features from Doppler shifts,and employ softmax function,Support Vector Machine(SVM),Support Vector Domain Description(SVDD)for mouth state identification,user authentication,spoofer detection,respectively.Finally,we design a balanced binary tree-based method to organize the trained binary classifiers and spoofer detectors for accurate user authentication.This work provides a novel pervasive security defense solution for mobile users.Second,this thesis studies a potential pervasive attack in mobile scenarios.This attack is an indirect eavesdropping side-channel attack,which can infer the keystrokes on QWERTY keyboards of touch screens merely levering the audio devices of a single smartphone.This work finds the acoustic attenuation principle could be utilized to localize the keystrokes on touch screens in an inconspicuous distance from users.Based on the principle,this work further combines the matched filter signal processing technique and geometry-based optimization method using finger movements to improve the successful rate for the attack.We first investigate the attenuation of acoustic signals,and find the finger during keystrokes can be localized through the attenuation of acoustic signals received by the microphones of a smartphone.Then,we utilize the attenuation principle to localize each keystroke,and further analyze the errors induced by ambient noises.To improve the accuracy of keystroke localization,we propose to track the finger movements during inputs via the phase changes and Doppler shifts of acoustic signals so as to reduce the errors of signal attenuation-based keystroke localization.Moreover,we design a binary tree-based search algorithm to infer keystrokes in a context-aware manner.Such a side-channel attack could be issued in various environments without the requirements of additional infrastructures.This work demonstrating potential mobile pervasive attacks aims to arouse the security awareness of users so as to improve the pervasive privacy security.Finally,combined with the previous researches on pervasive attack and defense for mobile scenarios,this thesis designs a pervasive security defense solution that can defend pervasive attacks,i.e.,vocal-tract behavior-based user authentication leveraging Frequency Modulated Continuous Wave(FMCW)techniques.This work investigates the vulnerability of lip reading-based user authentication to various pervasive attacks.To handle it,this work applies acoustic ranging techniques and transfer learning approaches to realize the attack-resisted and user-friendly user authentication system.This user authentication approach utilizes acoustic signals to sense the whole vocal-tract behaviors during speaking to realize a passphrase-independent user authentication for smartphones.We first apply the FMCW on acoustic signals to characterize the vocal-tract behaviors during speaking,and then employ Gaussian Mixed Model-Universal Background Model(GMM-UBM)to construct the passphraseindependent user authentication model with the extracted features of vocal-tract behaviors.Since the FMCW technique requires the synchronization between transmitted and received signals for accurate feature extraction,such FMCW-based user authentication system can resist various eavesdropping and mimic attacks in real authentication scenarios.Meanwhile,because of its passphrase-independent user authentication manner,the proposed system can provide user-friendly experience and support the applications in various fields other than user authentication.This work is inspired by the previous two researches,combining the revealing mobile pervasive attacks with the design experience of mobile pervasive security defense solutions.