Research On Privacy Protection For Distributed Computing

With the popularity of smart devices,more and more valuable data is generated.It is signif-icant to aggregate these data and give in-depth analysis.The traditional centralized analysis and computation framework may become incompetent due to the fast proliferation of data.An efficient alternative computation framework is distributed computing,where multiple server clusters collab-oratively fulfill a large-scale data analysis task.This computation framework not only mitigates the single-machine bottleneck in the centralized setting,but also has higher scalability and robustness to failures and faults.In recent years,it has received much attention from industry and academic communities,and many nations have established policies to promote its development.The data required for distributed computing may contain users' sensitive information.How-ever,the corresponding protection techniques are not appropriately addressed in some current dis-tributed systems,resulting in a serious situation of sensitive information disclosure.Thus,it is urgent to investigate the privacy-preserving distributed computing problems to reduce privacy dis-closure.Though there are many applications in distributed computing,most of them can be divided into two categories:Statistical information computation and machine learning.Moreover,by in-tegrating the two types of computation approaches,one could conduct other computation tasks.Hence,this thesis mainly tackles the privacy-aware problems regarding the above two distributed computing tasks.Some solutions addressing these issues have been proposed in the literature,but there are still some deficiencies and problems to be solved:1)It is lack of privacy-aware distributed maximum value computing approaches providing reliable protection;2)for the privacy-preserving distributed average computing problem,it is required to further quantify the dynamic property of the privacy guarantee and provide heterogeneous protections;3)regarding the privacy issues in distributed machine learning,the analysis of the accumulation of privacy losses over distributed it-erations is insufficient;4)to optimize the performance of privacy-preserving schemes,it is required to investigate the participation of data contributors in the mechanism design.Based on these ex-isting related works,we will conduct privacy preservation research on distributed maximum value computation,average computation,and machine learning.Further,we try to study the performance optimization mechanism including the participation of data contributors.The main contributions of the thesis are summarized as follows1.For the privacy-preserving distributed maximum value computing problem,we first prove that there does not exist a protection algorithm achieving exact maximum value computation and differential privacy simultaneously.We design a differentially private maximum con-sensus(DPMC)algorithm.It is proved that the DPMC algorithm preserves E-differential privacy,and we give the analytical expression of the resulted privacy-preserving level(P-PL).Moreover,we prove that the proposed algorithm achieves finite-time convergence,and analyze the performance in theory.2.Considering the privacy-aware distributed average computing problem,we propose a two-phase distributed computation framework providing heterogeneous protections.In the pro-posed framework,users obtain partial privacy control permissions,and servers execute three privacy-aware schemes in the average consensus iterations.The three schemes achieve d-ifferent privacy properties and computing performances.By employing Kullback-Leibler differential privacy,we obtain the PPLs in different phases and quantify the PPL of each iter-ation in the second phase.Then,we analyze the convergence guarantees and the computation accuracy of the three proposed privacy-aware schemes.3.Regarding the privacy-preserving distributed machine learning problem,we design an alter-nating direction method of multipliers based privacy-aware framework.A local randomiza-tion approach and a combined noise-adding method are leveraged to provide heterogeneous privacy protections depending on data's sensitive levels and servers' trust degrees.The per-formance of the trained model is analyzed according to the regularized generalization error.In particular,we give a theoretical bound of the difference between the generalization errors of the trained model and the ideal optimal model4.We then try to solve the conflict between privacy protection and computation performance in the privacy-aware distributed computing problems.Taking the average computation as an example,we design two performance optimization approaches using the incentive idea:User-side non-monetized incentive mechanism and server-side monetized incentive mecha-nism.In the non-monetized incentive mechanism,we derive the optimal reporting times;in the monetized incentive mechanism,we obtain the optimal combinations of noise variances and rewards.In the end,we summarize the thesis and discuss some future investigations.