Research On Distributed Asynchronous Optimization Algorithms For Skewed Data In Mobile Edge Computing

Distributed asynchronous offline training has received widespread attention in recent years.Compared with the synchronous architecture,the independent calculation and unconstrained characteristic of each computing node make asynchronous optimization present high performance and great efficiency on large-scale data and complex models.On the other hand,as mobile edge computing gradually becomes a trend,data is distributed from the cloud to the edge nodes.The feature of the edge nodes’ local data processing brings low latency and security,while distributed collaborative tasks have to face natural,nonindependent identically distributed(Non-IID)skew data.However,previous asynchronous methods do not have a satisfying performance on Non-IID data because the training process fluctuates compromising convergence.For distributed asynchronous optimization in mobile edge computing,a gradient scheduling algorithm with global momentum(GSGM)is proposed.Firstly,GSGM applies the momentum method on the central server instead of the edge nodes,in order to weaken the influence of the bias direction.Secondly,GSGM takes advantage of the scheduling strategy based on a white list to keep the gradient updates balanced and orderly.Finally,momentum acts on the partly average gradients,which adopts the average of the most recently calculated gradients at the end of scheduling as a good estimate of the unbiased direction.Experimental results show that under the same experimental conditions,GSGM on popular optimization algorithms can achieve a 20% increase in training stability with a slight improvement in accuracy on Fashion-Mnist and CIFAR-10 datasets.Meanwhile,when expanding distributed scale on CIFAR-100 dataset that results in more sparse data distribution,GSGM can perform a 37% improvement on stability.Moreover,only GSGM can converge well when up to 30 edge computing nodes,compared to the state-of-the-art distributed asynchronous algorithms.At the same time,GSGM is robust to different degrees of Non-IID data.