Research And Implementation Of Distributed ADMM Algorithm Parallel Model

Distributed parallel technology is an effective way to solve large-scale data processing.Alternating Direction Method of Multipliers(ADMM)is a universal parallel algorithm framework for distributed optimization.It provides solutions to the consensus problem and sharing problem in large-scale machine learning.The ADMM algorithm decomposes the original target problem into several subproblems and processes them in parallel,then coordinates the solutions of each subproblem to obtain the global solution.When designing a distributed ADMM parallel algorithm,there are mainly two parallel modes: data parallel and model parallel.Data parallelism is widely used and easy to implement,but for high-dimensional data,model parallelism can reduce the amount of calculation compared to data parallelism.Therefore,it is necessary to choose a suitable parallelization mode for different problems.At the same time,in the distributed environment,the communication overhead between different nodes of the ADMM algorithm is relatively large,and in practical applications,the distributed ADMM algorithm has a slower convergence speed.The main work of this article focuses on the following points:1)According to the characteristics of the distributed ADMM algorithm and the communication bottleneck in the multi-core cluster operating environment,a new hierarchical communication structure is designed to improve the communication efficiency of the algorithm.2)By analyzing the effect of the global consensus ADMM algorithm(based on data parallel)subproblem solution on the convergence speed of the algorithm,combined with the hierarchical communication structure,a strategy for dynamically scheduling the ADMM subproblem optimization algorithm is designed.At the same time,based on the convergence conditions of the algorithm,we choose the primal residual and dual residual as the basis for selecting the optimal algorithm for each scheduling and design two different dynamic scheduling algorithms.The experimental results show that the dynamic scheduling strategy can accelerate the convergence speed of the algorithm.3)Based on the analysis of empirical minimization problems based on feature partitioning,combined with shared optimization problems(data and features can be divided),a feature-based distributed ADMM sharing algorithm(based on model parallel)is proposed to solve high-dimensional models.By dividing the high-dimensional data into multiple low-dimensional data according to the characteristics,and dividing the high-dimensional model into multiple submodels,then distributed to multiple nodes for parallel processing to reduce the computational overhead caused by high-dimensional data.This paper shows the convergence and iteration complexity analysis and verifies the performance of the algorithm through experiments.In this paper,the parallel algorithms are tested on ZiQiang 4000 cluster system of Shanghai University.The experimental results show that the hierarchical communication structure can effectively reduce the communication overhead of the algorithm.Solving the sub-problems using the dynamic scheduling strategy can speed up the algorithm's convergence speed.The feature-based distributed ADMM algorithm can improve the convergence speed and reduce the running time of the algorithm in the tolerate range of the loss of accuracy when solving high-dimensional models.