Research On Lightweight Personalized Federated Learning For IoT Devices

With the popularization of Internet of Things applications,the explosive growth of massive data promotes the development of high-quality services.The efficient analysis of data by artificial intelligence drives the transformation from digital service to intelligent service.Federated Learning protects the data privacy of devices by exchanging model parameters between devices and servers.Due to the statistical heterogeneity of the data of Internet of Things devices,the training quality of local models is different.In addition,the differentiated computing and communication capabilities of devices will also affect the local training and data transmission delay,and reduce the training efficiency of Federated Learning.This thesis starts from the perspective of personalized model,based on the theory of contrastive learning and model compression,the quality and efficiency of model training are improved.Specific work is as follows:Aiming at the difference of training quality of local models caused by statistical heterogeneity of data,a hierarchical personalized federated learning algorithm based on contrastive learning is proposed.In the local training stage,the connection with the global model is strengthened and the performance of local model learning is improved through the contrastive training of this model and the previous model? in the server aggregation stage,each model is adaptively weighted based on the extent to which other models benefit and then sent locally?training and aggregation are performed alternately locally and on the server and ultimately generate personalized models for each device.By conducting experiments on simulated and real data,the proposed algorithm is 20% to 50% higher than the federated average on model accuracy,6% to 12% higher than typical algorithms such as federated multi-task learning and federated clustering,and has good convergence.Aiming at the problem that the differentiated computing and communication capabilities of devices reduce the training efficiency of federated learning,a personalized federated learning algorithm based on model quantization is proposed.In the local training stage,the quantization of the model is combined with the loss function,and the loss function and the quantization center related to the device are optimized at the same time,and the quantization of the model in the final deployment can achieve lightweight inference without reducing the prediction accuracy? in the upload stage,each device only uploads the quantized center value and the index corresponding to each parameter,which can greatly reduce the transmission traffic compared to uploading the complete model parameters? the two stages iterate each other until convergence.Through experiments on simulated data and real data,the average interactive traffic between the device and the server during the training process can be reduced by 79%,the convergence speed is close to uncompressed,and the model accuracy is only slightly reduced by 2% to 5% compared to the uncompressed personalized learning,enabling lightweight communication.This thesis mainly proposes two federated learning algorithms,which are used for personalized model training and model compression respectively.The proposed algorithm can reduce the network load and transmission delay.The experiment proves that the algorithm in this thesis can improve the efficiency and accuracy of federated learning,and has good convergence and stability,and finally achieves lightweight personalization federated Learning.