Distributed Conjugate Dual Gradient Algorithms

Distributed optimization is an optimization method for a large number of autonomous individuals(computing agents)to solve the optimization problem of the whole system or network through local information transmission or interaction.Due to the need for agents to interact local information with their neighbors,unnecessary privacy leakage may inevitably occur;Among these massive data,the majority is dynamic stream data with real-time update characteristics,and it is difficult for traditional batch processing methods to effectively process these dynamic stream data.Therefore,for the above two problems,the privacy protection of distributed conjugate dual gradient algorithm and the distributed online learning algorithm for real-time processing of stream data are studied.Firstly,aiming at the problem of the privacy leakage including the states and local function caused by the exchange of information between agents,a privacy-preserving distributed optimization algorithm based on the distributed conjugate dual gradient algorithm and effectively combined with the Paillier Cryptosystem mechanism in the homomorphic encryption technology-the privacy-preserving conjugate dual gradient algorithm is proposed,and the algorithm's convergence is proved in the undirected time-varying network topology and the local cost functions are strongly convex functions;Further theoretical analysis proves that one agent cannot obtain the sensitive information of its neighbors even by the intentional collection of the multi-step intermediate information,and thus the proposed algorithm can effectively ensure the privacy protection of agents.Then,the privacy protection of the algorithm is verified by numerical simulation.Secondly,in the time-varying and strongly connected networks,aiming at the problem of real-time processing of stream data in dynamic environment,a distributed online learning optimization algorithm based on the distributed conjugate dual gradient algorithm—distributed online conjugate dual gradient algorithm is proposed by adding online setting to the distributed conjugate dual gradient algorithm,establishing the related mathematical model and iteratively solving the optimization problem.Then we give the Regret bound of original variable and the dual variable of the proposed algorithm.The convergence of the algorithm and the sublinearity of the Regret bound with respect to time T are proved by the theory when the local cost functions are strong convex functions.In suumary,aiming at the problem of privacy leakage in the distributed computing process,a privacy-preserving distributed conjugate dual gradient algorithm is proposed with homomorphic encryption technology to ensure data privacy and we prove the convergence of the algorithm and the effectiveness of privacy protection by theoretical analysis.Aiming at the problem of real-time processing of stream data among the massive data,a distributed online conjugate dual gradient algorithm is proposed,and we prove the convergence of the algorithm by the theoretical analysis.Figure[12]Table[5]reference[64].