| With the explosive growth of the data generated at the edge of network,numerous computing tasks have moved from the cloud to the edge.However,the processing capability of edge nodes is limited.The edgecloud collaborative architecture,which combines the advantages of edge computing and cloud computing,has become a popular computing paradigm under the trend of interconnection of all things.In the edge-cloud collaboration system,to accelerate the intelligentization process at the edge,deep learning technologies are widely used for the computation and processing of massive data.Nevertheless,the complex calculations and huge storage demands of deep learning pose enormous challenges for deployment in resource-constrained edge environments.Knowledge distillation(KD)can effectively address this issue by compressing models and reducing resource consumption while almost not damaging model performance.This paper conducts research based on the edge-cloud collaborative architecture from two perspectives:knowledge distillation for model compression and optimized collaborative model training with data security.The main research contents are summarized as follows.(1)Multi-scale feature-based edge distributed online knowledge distillation.In response to the limitations of traditional knowledge distillation,which heavily relies on pre-trained teacher models and has limited information transfer capabilities,this paper proposes a distributed online distillation scheme for edge environments,which reduces the difficulty of deploying deep models on edge nodes.This approach breaks the pre-established strong and weak relationship between teacher and student models and allows the transmission of informative intermediate feature knowledge,significantly improving the multi-scale representation ability of the model.Moreover,the approach enhances the focus on important channels and regions in feature maps.Ultimately,the method generates enhanced features by fusing global information,which is used to guide the student models to learn high-quality feature knowledge.Experimental results show that the proposed approach can considerably reduce the computational complexity and minimize the accuracy loss caused by model compression,while being compatible with model heterogeneity scenarios.(2)Data-free enhanced federated knowledge distillation based on edge-cloud collaboration.In response to the conflict between the data exchange involved in knowledge distillation and the demand for local data security protection at various edge nodes in real-world scenarios,this paper proposes a federated learning(FL)based hierarchical distillation scheme,which can achieve superior collaborative training among edge nodes through hierarchical distillation without sharing data.Firstly,this paper addresses the problem of common dataset dependency in federated distillation scheme based on generative adversarial network and the cloud aggregates global distributions through server distillation.Secondly,virtual data is generated based on global knowledge at the local level,allowing effective knowledge to circulate locally in a data-free way,thus mitigating the accuracy degradation caused by data heterogeneity in edgecloud collaboration scenarios.Experimental findings indicate that the proposed approach effectively utilizes global knowledge with protecting user data security,and enhances the robustness and generalization performance of the models while maintaining lightweight architectures. |
