Research On Theories And Algorithms Of Gradient Perturbation And Alternating Minimization For Differential Privacy Protection

With the widespread of network information,the rapid development of related technologies,and the gradual popularization of digitalization,the collection of user data becomes more and more convenient,resulting in the rapid growth of user data and gradually accumulating.As one of the most important core assets,user data provides important analysis resources for science and technology companies,scientific research institutions,and government management departments.By mining and analyzing these user data,it can not only quickly and accurately discover the changes of users' demands,but also forecast the market development trends,which will assist enterprises,government agencies,and other departments in making timely strategic adjustments to satisfy market demand and promote economic growth.The development of artificial intelligence and other related technologies provides strong support for data analysis.The analysis and processing of user data possessed by machine learning algorithms and data mining technologies can obtain a wealth of valuable information,thereby improving service quality and accelerating product upgrades in order to improve users' experience.However,because user data generally contains sensitive information,the extreme development and utilization of such data put users' sensitive information facing a huge risk of being leaked.Therefore,how to protect users' privacy while fully mining and utilizing users' data information is a crucial scientific problem that needs to be solved urgently.The goal of differential privacy data processing is to obtain sufficiently accurate learning models based on learning algorithms that are utilized to exploit user data without leaking users' privacy.In order to effectively evaluate the performances of learning algorithms,a series of differential privacy learning models and algorithms are constructed based on empirical risk minimization techniques,so as to achieve the goal of differential privacy data processing.The main contributions of this thesis are summarized as follows:1.For the problem of differential privacy data analysis,this thesis focuses on the design and implementation of the differential privacy algorithm under gradient perturbation.Inspired by the Barzilar-Borwein(BB)method,a stable BB gradient perturbation differential privacy algorithm was proposed.Because this algorithm does not involve any parameters in the determination of BB step size,so it avoids some difficulties and troubles arising from the adjustment of parameters,as well as the negative inuence of unsuitable parameter selections on calculation results of solving algorithms,which greatly improves the computational efficiency.Besides,both the differential privacy guarantee and the accuracy guarantee of the proposed algorithm were analyzed and proved theoretically.Furthermore,a series of numerical experiments were carried out to verify the validity of the proposed algorithm based on two classical machine learning problems,namely Logistic Regression(LR)and Support Vector Machine(SVM).2.For the problem of the gradient perturbated differential privacy algorithm design in differential privacy data analysis,a privacy budget adaptive algorithm was proposed based on the Armijo step rule.Because the proposed algorithm does not require determining the total iterative steps in advance,it avoids the inuence of the improper total iterative steps on the algorithm.Moreover,the proposed algorithm can adaptively determine the privacy budget in each iteration.As a result,it improves the rationality of privacy budget allocation.In addition,the Armijo line search technique can save the cost of privacy budget through the whole iterative process,thus it improves the efficiency of the algorithm.Both the differential privacy guarantee and accuracy guarantee of the algorithm were analyzed and proved theoretically,and the effectiveness was further verified by numerical experiments.3.For the learning task of differential privacy data analysis and publishing,a dualmode empirical risk minimization learning model was proposed based on the bilevel programming;Then,an alternative privacy minimization algorithm was established to solve the above two-mode learning model.In particular,the proposed algorithm alternately updates the model parameter and the synthetic dataset during the iteration process.It can output the learning model and publish the synthetic dataset that meets the privacy protection simultaneously.Further,the convergence,privacy,and accuracy of the proposed algorithm were theoretically analyzed.Finally,the effectiveness of the proposed algorithm was verified by numerical experiments.