The Research And Implementation Of Distributed Deep Learning Optimization Methods For Edge Computing

With the development of deep learning technology,the demand for data and computing power in the training process of deep learning model is increasing year by year.Distributed deep learning technology is an effective way to solve the problem of large-scale model training based on massive data by organizing multiple computing devices to complete model training cooperatively.In recent years,the number and computing power of mobile devices and IOT devices are growing,the public's awareness of data security and privacy protection is improving,and the distributed training of deep learning on edge terminal devices has been widely concerned.However,due to the high cost of communication in the edge scene,the performance and bandwidth of terminal devices are inconsistent and vulnerable to the impact of the environment,there are still many challenges to carry out efficient distributed model training.Aiming at the existing problems of distributed model training of deep learning in edge scene,this paper proposes gradient compression method based on multi-resolution grid clustering and parameter synchronization update method based on adaptive delay from two aspects of reducing parameter traffic and optimizing parameter update process to improve the efficiency of distributed deep learning training.Gradient compression method based on multi-resolution grid clustering,through the analysis of the characteristics of parameter gradient,adopts the idea of grid clustering to divide the gradient into multiple intervals according to its absolute value.Then it uses the average value of the interval as the representative value of the gradient,and only transmits the representative value and index when transmitting,so as to reduce the amount of gradient data.Through the analysis of the experimental results,this method can ensure the accuracy of the model,at the same time,on the basis of the threshold compression method,the amount of transmitting gradient can be reduced by nearly half,which can effectively reduce the communication time in low bandwidth scenarios.The parameter synchronization updating method based on adaptive delay defines the delay index,adopts the strategy of limited delay waiting and discarding overdue updates.It optimizes the parameter updating process under distributed training,and adjusts the computation or communication traffic in the model training process according to the state of edge terminal equipment,so as to reduce the influence on the training time and accuracy of the model caused by the heterogeneity of equipment and the complexity of environment.The experimental results show that the proposed method performs better than the traditional methods in adapting to heterogeneous devices and complex network environment,and can effectively improve the training efficiency of the model.On the basis of the above two methods,this paper also designs and implements a distributed deep learning system for edge computing.The system has the ability of distributed model training on multiple terminal devices,and provides the functions of device management and model training management.By organizing several raspberry pie devices for distributed training,the effectiveness of the system and the feasibility of the application of distributed deep learning on edge terminal devices are verified.