| Federated learning decomposes centralized learning tasks to network terminals.It can directly utilize the computing power of terminal devices to train local models,and shares local models among terminal devices.Therefore,federated learning can protect user data privacy and reduces communication overheads caused by the transmission of raw device data.However,communication and computing problems of federated learning bring challenges when referring to its practical applications.In order to further facilitate the application of federated learning,it is necessary to study the communication and computing problems of federated learning.This dissertation focuses on computing problems of centralized federated learning and communication problems of decentralized federated learning.A computing optimization algorithm of centralized federated learning is designed to accelerate convergence and reduce computing costs,and data compression and quantization algorithms of decentralized federated learning are designed to reduce communication costs.The main research contents and contributions of the dissertation are summarized as follows:(1)Research on acceleration method for centralized federated learning.The existing centralized federated learning algorithm adopts Gradient Descent(GD)to conduct local model training.The gradient information from the current iteration is used to update the model,which is easy to cause gradient oscillation and slows the convergence rate of the algorithm.In order to eliminate Gradient oscillation and accelerate convergence of centralized federated learning algorithm,the historical gradient information from previous iterations is considered to be used for model updating of the current iteration.Therefore,we design a centralized federated learning algorithm based on Momentum Gradient Descent(MGD).In the algorithm,MGD is adopted for local model training,and model parameters and momentum parameters need to be updated in each iteration.By theoretical analysis,we prove the convergence of the algorithm,obtain its sub-linear rate of convergence and derive its convergence upper bound.By comparing the convergence rate of the algorithm with that of centralized federated learning algorithm,we find that the convergence rate of the algorithm is faster under certain conditions.Simulation results based on real-world datasets demonstrate the effectiveness of the proposed algorithm in accelerating convergence and decreasing computing costs.(2)Research on data compression method for decentralized federated learning.Aiming at communication problems of decentralized federated learning,we study a compression method for model parameters in the process of model exchange.Increasing the computing frequency can improve communication efficiency but decreases model consensus,while increasing communication frequency causes inferior communication efficiency but improves model consensus.In order to jointly consider computing and communication to balance model consensus and communication efficiency,we design a general decentralized federated learning algorithm with communicating and computing multiple times.Based on this algorithm,we design a compression algorithm for model parameters to improve communication efficiency.We prove that the general algorithm is globally convergent,derive a convergence upper bound about communication frequency and computing frequency.Results of theoretical analysis show that increasing communication frequency can improve convergence rate,and increasing computing frequency will reduce the convergence rate.In order to improve communication efficiency,we design a decentralized federated learning algorithm with model parameter compression.Results of theoretical analysis show that compressing model parameters will reduce convergence performance of decentralized federated learning.Simulation results based on real-world datasets verify the influence of communication and computing frequency on the convergence rate of the general decentralized federated learning algorithm,and show that the model parameter compression algorithm can effectively improve communication efficiency.(3)Research on quantization method for decentralized federated learning.Focusing on communication problems of decentralized federated learning,we study a quantization method for model parameters.Convergence rate of decentralized federated learning is nonlinearly decreasing.Values of loss function decrease rapidly in the initial training process while declining slightly in the end.Quantizing model parameters by means of a fixed number of quantization levels will enable devices to transmit more bits in the training process.Distribution of model parameters is non-uniform and dynamic.Quantizing non-uniformly and dynamically distributed model parameters by uniform quantization will result in a large quantization distortion.We first design a decentralized federated learning algorithm with adaptive distribution of quantization levels.The algorithm can adjust the quantization levels adaptively to match distribution of model parameters and reduces quantization distortion.We derive an upper bound of quantization distortion and convergence of the algorithm.The algorithm has a sub-linear convergence rate.Theoretical analysis results show that the proposed algorithm reduces quantization distortion and improves convergence performance.According to the convergence upper bound of the algorithm,we derive the optimal number of quantization levels about iterations,which increases with iterations.Based on the optimal number of quantization levels,we design a decentralized federated learning algorithm with adaptive number of quantization levels.The simulation results based on realworld datasets show that the algorithm of adaptive distribution of quantization levels effectively reduces quantization distortion and improves the convergence rate.When achieving targeted training loss,the algorithm of adaptive number of quantization levels can effectively reduce the number of communicated bits. |
