Research On Robust Communication And Computing Design For Federated Learning

Without requiring the upload of raw samples from the edge devices,federated learning can collaborate with the edge devices and the central server to complete model training.It protects the privacy of the edge devices,and reduces the latency of uploading numerous raw samples.Nevertheless,the robust problem in federated learning brings some new challenges.It is caused by the inaccurate results from model aggregation and sample collection.Moreover,it will result in serious effect on federated learning,for example,compromising the performance of model and declining the convergence rate of training.Therefore,it is essential to solve the robust problem in federated learning.In this dissertation,we consider both communication and computing design for robust federated learning with model error and label noise.The objective of the robust design is to guarantee the accuracy of model aggregation results and improve the performance of model.The main contributions of this paper are summarized as follows.Firstly,we design robust transceiver to solve the model error,which is caused by the imperfect channel state information(CSI)in the model aggregation.The key issue is to decrease the mean square error(MSE)to maintain the accuracy of results from model aggregation.To be specific,it is shown that the transceiver training process of collecting individual CSI destroys the advantage of federated learning.The equivalent CSI is developed to handle this problem via the simultaneous transmission of the pilot from each edge device.The corresponding transceiver training process is also proposed based on the equivalent CSI.Furthermore,robust transceiver designs with channel uncertainties are provided for both individual and equivalent CSI separately.Transceiver with the imperfect individual CSI can be optimized by utilizing an iterative method.The computational complexity of the algorithm is related to the number of edge devices.Meanwhile,the receiver design with imperfect equivalent CSI is transformed into a convex optimization problem via the S-procedure,and the transmitter design has a closed-form solution.The computational complexity is merely relevant to the number of antennas at the central server.The MSE improvement and complexity reduction of the proposed designs are demonstrated via simulation.Additionally,robust model training is developed in federated learning,when both edge devices and central server can only obtain the imperfect acquisition of the model.The main idea is adding the features of model error into the design of loss function.The model error is occurred at the steps of local model uploading and global model broadcasting,and will be superimposed by two parts.There are some differences between the robust design for noise and quantization error,although both of them will generate the model error.Specifically,noise can be described by a statistical model,and its statistics are used to devise the regularized loss function.The algorithm converges at a linear rate,where the bigger variance of noise,the slower convergence rate.A deterministic model is employed in modeling quantization error.The model training first samples the quantization error by the sampling average approximation method,and then calculates the loss function via the successive convex approximation scheme.The algorithm also has a linear rate,and it can be accelerated by reducing learning rates.Simulation results show that the proposed designs have the advantage of prediction accuracy with numerous edge devices.Finally,the robust trainings for different kinds of data distributions are considered to deal with the noisy labeled data in federated learning.We aim at assigning the weights of each loss function,and computing the model and weights in collaboration with the edge devices and the central server.Two kinds of data distribution are considered and referred to as full-data and part-data respectively,where the processing of noisy labeled data is different.Particularly,the edge devices contain noisy samples and a bit clean samples with full-data,while in federated learning with part-data,some of the edge devices do not contain clean samples and the weights assignment can not be processed.A training scheme based on the dual decomposition method is proposed with full-data,which includes the training of model,the model aggregation method,and the computing of weights.The algorithm has a sub-linear convergence rate,which is positive correlation with learning rates.Also,a training method with part-data is designed by utilizing the alternating direction multiplier method.The edge devices without clean samples only need to compute model.The others complete the calculation of both weights and models,and help those edge devices without clean samples to process the weights assignment.The proposed design still has a sub-linear convergence rate,which is a feasible rate in federated learning.Simulation results illustrate that both proposed training processes improve the prediction accuracy under various conditions,and the algorithms are more capable with a small number of clean samples.