Research On Speech Data Protection Based On Differential Privacy

With the rapid development of modern technology,speech recognition and intelligent voice assistants have been widely used in daily life.However,speech service providers collect and store users’ speech data and may share it with third parties.Speech data contains two aspects of personal privacy information: semantic information and biometric characteristics.From the perspective of semantic information,speech may contain sensitive content such as names,addresses,and diseases.From the perspective of biometric characteristics,the voiceprint information contained in speech can identify a specific individual’s identity.Attackers can use speech technology to identify the speaker’s identity and sensitive semantic content in speech data,causing serious privacy breaches and threatening the user’s personal and property security.To address the privacy leakage of speech data,this paper proposes a speech data privacy protection mechanism based on differential privacy technology,which protects users’ speech data privacy from two aspects: speech content and acoustic features.Specifically,this paper uses sensitive words recognition method based on semantic similarity and sensitive words replacement method based on differential privacy to identify sensitive words in users’ speech content and replace them with non-sensitive words.At the same time,the differential privacybased voiceprint anonymization method and multi-speaker speech synthesis technology are used to convert the user’s voice into another person’s voice.After the privacy information is removed,the speech still has high speech quality and semantic understandability,and can be safely shared with third parties or used for downstream task training.The main research contents and contributions of this paper are summarized as follows:(1)A differential privacy-based method for filtering sensitive speech content is proposed to address the needs of identifying and desensitizing sensitive semantic information in speech data.This approach detects sensitive words in speech content based on semantic similarity and utilizes utility-optimized local differential privacy to replace sensitive words with words that are more semantically similar with a higher probability.Experimental results demonstrate that this method can effectively identify and filter sensitive words in speech content while preserving data utility and protecting the privacy of speech content.(2)A differential privacy-based method for acoustic feature privacy protection is proposed to address the need for anonymizing voiceprint information in speech data.This method utilizes a differential privacy-based voiceprint anonymization approach,using the exponential mechanism to anonymize users’ voiceprints,and synthesizes speech waveforms by combining the anonymized voiceprints with desensitized speech content using a multi-speaker speech synthesis method.Experimental results demonstrate that this method achieves both voiceprint privacy protection and data utility preservation,as the synthesized speech retains good semantic understandability and naturalness.(3)Based on the above two research parts,a differential privacy-based privacy protection system for speech data is designed and implemented.The requirements of the speech privacy protection system are analyzed,and the functionality modules including recording,speech visualization,sensitive speech content filtering,and voice anonymization are validated,showcasing the relevant pages of the system.This system can remove speaker identities and sensitive speech content from speech signals while maintaining the quality of the speech data.