Research On Differential Privacy-preserving Algorithms In Convolutional Neural Networks

With the development of big data,it promotes the deep learning algorithms widely used in real-world,such as biomedicine and facial recognition.Convolutional Neural Networks(CNN),a highly efficient recognition method that is widely valued in deep learning,can effectively process a large number of high-dimensional images.However,attackers with different background knowledge gain benefits from two aspects.One is directly obtaining sensitive information of the original data.The other is indirectly extracting the parameters of the convolutional neural network.Convolutional neural network faces the privacy leakage and security risks that directly obstacle its development.Differential privacy effectively protects sensitive data information,is backed by rigorous mathematical proofs and is lightweight.However,how to obtain the balance between privacy and utility by applying differential privacy techniques to convolutional neural networks is a problem that needs to be studied at present.Moreover,due to the randomness of noise,the parameters may oscillate repeatedly around the optimal value,which affects the model convergence.To achieve an effective trade-off between privacy and precise of deep learning models and to ensure model convergence,the following three studies are conducted in this paper.(1)To achieve the balance between privacy and utility,adaptive algorithm for gradient noise addition is proposed in this paper,because the gradient change of the convolutional neural network is a random to fine-tuned process,so the privacy budget of the perturbation process is adaptively assigned;meanwhile,the regularization term is introduced to avoid the noise is too large.Combined with the characteristics of the model and the reasonable budget assignment can achieve a better balance between privacy and utility.(2)In order to meet the privacy and utility needs of different users,this paper proposes a differential privacy utility optimization algorithm,which analyzes the privacy and utility of the model from a quantitative perspective,then privacy-utility balance problem is solved optimally using the Lagrange multiplier method under constraints.Finally,the appropriate parameters are selected within the optimal upper and lower bounds of the resolved noise variance,depending on the need for privacy and utility.The balance of privacy and utility can be implemented on the basis of quantification,and the availability of the model is guaranteed.(3)To solve the model convergence problem due to privacy protection,a stochastic gradient descent algorithm is proposed based on Polyak adaptive step size.On the basis of privacy protection,combining Polyak steps for the loss function,the constraints of the Passive-Aggressive(PA)algorithm are extended nonlinearly to make the algorithm applicable to convolutional neural networks,so as to obtain the step size parameters based on the sample gradient values and loss values;at the same time,a relaxation variable is introduced into the constraints to control for the trade-off between different sample losses and make the algorithm more suitable for general models and capable of stable updates.Finally,the adaptive step size adjustment reduces the impact of privacy protection and ensures better convergence of the model.