Research And Application Of Parallel Split Learning Algorithm Based On Distillation Technology And Incentive Mechanism

With the rapid increase in the number of communication equipment and sensing equipment,distributed machine learning technology is broadly used in edge computing,smart cities,intelligent vehicles and other fields.However,distributed machine learning algorithms such as split learning still have many problems in practical applications.First,due to the characteristics of serial training,the training time of split learning increases significantly with the increase of the number of devices,which will make it impossible to deploy split learning in large-scale application scenarios with thousands of communication devices.Secondly,due to the differences in the regions and time zones where the devices are located,the distribution of data on different edge devices is often not the same,which causes the accuracy of the models trained by aggregating the unevenly distributed data on these devices to decrease greatly.To solve these problems,this paper first proposes a parallel split learning algorithm based on distillation technology,which slightly improves the accuracy of the model while significantly reducing the training time;then an incentive mechanism is designed to make the proposed algorithm better adapt to the training environment with uneven data distribution on the user devices;finally,the application of the proposed algorithm in the data fusion of unmanned vehicles is considered.The main contributions of this article are as follows:1.We propose a parallel split learning algorithm based on distillation technology.Through the parallel training of the models on the edge devices,the algorithm greatly reduces its training time under the premise of protecting data privacy;a distillation loss function is used to replace the synchronization process of the models on the edge devices,which enables the model on the edge device to learn the knowledge of other edge devices,which improves the accuracy of model classification;and the convergence of the proposed algorithm is proved.2.We design an incentive mechanism based on Stackelberg Game to encourage users to use more evenly distributed data to participate in training.Firstly,a Stackelberg Game model is established based on the data distribution of edge devices.Then we prove the existence of the Nash equilibrium solution of the game model.Finally,a grid search algorithm is designed to find the optimal solution.In scenarios where data is unevenly distributed on edge devices,parallel split learning algorithms combined with the incentive mechanism can greatly improve the accuracy of model classification.3.We apply the proposed parallel split learning algorithm to the road detection task based on data fusion.Firstly,the road detection model is analyzed to find the optimal split point in the model.Then a distributed road detection method is proposed,and the incentive mechanism and distilled loss function are adjusted to fit the road detection model.The experimental results show that compared with other distributed machine learning algorithms,the training time of the proposed algorithm is reduced by 66% on average,and the accuracy of model classification is improved by 7% on average.At the same time,in scenarios where data on edge devices are unevenly distributed,the training scheme based on the incentive mechanism can increase the accuracy of model classification by an average of 125%.In the road detection task,the maximum F1 score of the proposed method is improved by an average of 1.3 points.