Research On Privacy-Preserving Deep Learning Based On Differential Privacy

In recent years,deep learning based on artificial neural network has been developed rapidly and been widely used in various fields.The success of deep learning cannot be achieved without the support of massive training data.However,there are obvious privacy problems in the huge amount of data required for deep learning.With the improvement of people's privacy protection awareness and relevant national laws,the privacy problem in deep learning seriously restricts the further development of it.Privacy protection for deep learning can not only protect sensitive information,but also promote the largescale application of deep learning,which has very important theoretical and practical significance.Collecting data on a large scale to train the learning model locally is called centralized learning.Because of the abundant data,this method can achieve good learning effect.Distributed collaborative learning can unite multiple participants with insufficient available data.It can achieve better learning results than individual training through mutual cooperation without directly sharing sensitive data.In this thesis,we study the privacy security of centralized deep learning and distributed collaborative deep learning.Aiming at the shortcomings of the existing algorithms,two privacy-preserving deep learning algorithms based on differential privacy are proposed.The main contents are as follows:1.In a centralized learning environment,we propose a new basic deep learning model that satisfies the differential privacy guarantee,which is called Differentially Private Deep Learning Based on Analytic Gaussian Mechanism and Functional Mechanism.Its design is based on the quantizability,composability and post-processing invariance of differential privacy.That is,the desired system is approximated by a combination of functions of bounded sensitivity.The main steps of the algorithm are as follows: The first step is to use the Analytic Gaussian Mechanism to disturb the input layer of the model;Secondly,several hidden layers are constructed on this basis;The third is to use the extended Function Mechanism to interfere with the loss function of deep learning.It has the advantages of less model complexity and simple privacy budget calculations.Compared with existing algorithm,more accurate classification results can be obtained on the basis of the same privacy guarantee in the classification task.2.In a distributed collaborative learning environment,we design a new distributed collaborative deep learning algorithm based on differential privcy.It is applied to scenarios in which the participants' local data sets have the same feature space,but with different samples.Under the coordination of the central server,participants use the distributed collaborative communication protocol based on the Piecewise Mechanism to carry out synchronous upload,secure aggregation and download processes.The algorithm adds differential privacy interference to the shared information of learning participants.While protecting the privacy of participants,it can overcome the problems of small local data set and single sample to the greatest extent and improve the learning effect of participants.This algorithm is more stable than the traditional differential privacy Laplace Mechanism,and does not require the central server to maintain global parameters.This further improve the level of privacy protection.