Research On Deep Learning Methods To Support Privacy Protection

In the era of big data,it has become a common phenomenon to use massive data to support various decisions.And deep learning has received widespread attention for its advantages such as end-to-end learning and the ability to achieve relatively high task accuracy.Along with the rapid development of deep learning,a series of privacy protection issues have occurred,ranging from the illegal sharing of personal data to the inference attacks based on the public model and parameters,which has caused serious privacy leakage.Therefore,it is of great significance to study a deep learning method that can protect the privacy of training data while ensuring relatively high data utility.Differential privacy is a perturbation-based privacy protection technology.Applying it to deep learning enables the attacker to infer only the perturbed data,and achieves the purpose of privacy protection,at the same time,the appropriate amount of perturbation does not significantly reduce data utility.Aiming at the problems of image generation and non-Euclidean structure data in deep learning,we conduct research on privacy protection deep learning methods by combining with differential privacy.The main contributions of our work are concluded as:(1)Aiming at the possible privacy leakage risk of training data,we improve the deep learning methods by combining with differential privacy.For image generation task,we propose a multi-generator generative adversarial network based on gradient clipping and perturbing.And for semi-supervised classification task of nodes in the graph,we propose a functional mechanism-based graph convolutional neural network.On the premise of protecting the privacy of training data,we ensure that the above models have relatively high data utility.(2)We propose a multi-generator generative adversarial network based on gradient clipping and perturbing,namely DP-MWGAN,to solve the problem of privacy-protected image generation.Firstly,a multi-generator architecture and parameters sharing method are adopted,and the EM distance is used as loss function to construct a multi-generator generative adversarial network MWGAN.Secondly,considering the privacy protection of training data,a perturbation scheme based on differential privacy is designed,which consists of three steps:adaptive gradient clipping threshold determination,gradient clipping and Gaussian noise perturbation.Applying the scheme to the training steps,we can finally get a multi-generator generative adversarial network DP-MWGAN,which can both perform image generation task and achieve privacy protection.Finally,experiments were performed on the CIFAR-10 image dataset,the experimental results show that the proposed MWGAN model is better than WGAN and WGAN-GP according to the comprehensive results of Inception Score and EM distance.And the DP-MWGAN can protect the privacy of training data while not significantly reducing the Inception Score,ensuring a relatively high similarity of the original image.(3)We propose a functional mechanism-based graph convolutional neural network,DP-IGCN,to solve the semi-supervised classification task of nodes in graph.Firstly,referencing the traditional CNN models that use fully connected layer to learn the high-level features,we improve GCN model and propose IGCN,add fully connected layer to extract high-level features of graph data.Secondly,considering the privacy protection of training data,a perturbation scheme based on function mechanism is designed.Specifically,an approximate polynomial of the loss function is derived through Taylor decomposition,and Laplacian noise is added to the polynomial coefficients for random perturbation.The perturbed loss function is applied to the training process,and we can finally get a graph convolutional neural network DP-IGCN,which can perform graph node classification tasks and achieve privacy protection.Finally,experiments were performed on three datasets,Cora,Citeseer,and Pubmed.The experimental results show that the proposed IGCN model has higher node classification accuracy than the original model,and the DP-IGCN can ensure the privacy of training data while ensuring a relatively high graph node classification accuracy.