| The increasing popularity of Internet of Things(IoT)is driving the development of diverse applications,e.g.,interactive online gaming,face recognition,3D modeling,VR/AR and vehicle networking systems,that generate amounts of data from physical worlds each day.In order to improve service quality for users,it is necessary to use these data efficiently for model training and knowledge inference,so as to realize the deep integration of Internet of Things and artificial intelligence.Under traditional application scenarios,the huge amount of application data will be transmitted to the remote cloud platform across the core networks.However,the burdened heavy load of current core networks make the cloud response to the application requests with a longer delay,which not only affects the model training efficiency,but also reduces the user experience,and increases the risk of user privacy disclosure.To this end,edge computing is proposed as a new paradigm.The main idea of edge computing is to use the computation capacity edge devices in network,so that the data and tasks can be processed on the edge devices nearby instead of being uploaded to the cloud,which not only alleviates the load pressure of the core network,but also provides users with low latency and high security services.However,compared to centralized data centers,edge computing faces some new challenges to implement highly efficient model training and knowledge inference.1)Edge heterogeneity:Various edge nodes with diverse geographic locations,computation capacity,network connections,data volume and data distribution,may perform training or inference tasks.As a result,the quality of accomplishment of tasks(e.g.,computation delay,communication delay,energy consumption,model accuracy)may vary greatly.2)Resource constraint:The computing resource of edge nodes is usually limited,so computation-intensive tasks(such as model training)may cause high computation delay,while the communication resource is usually limited in edge networks.As a result,the transmission of data-intensive tasks(such as VR/AR)may cause high communication delay.3)Robustness requirement.Numerous devices.The edge network contains a large number of nodes,and it is a great challenge to ensure system robustness by designing effective mechanisms to schedule these nodes cooperating in model training or knowledge inference.4)Environmental dynamics:The edge devices may be in a high-speed and unpredictable state,and the data traffic of the edge network is also in dynamic change.In this dissertation,to address the challenges of the above,the model training is considered from two aspects of updating mechanism and architecture,and a semi-asynchronous federated learning mechanism and a unified hierarchical federated learning clustering mechanism are proposed,respectively.After the trained models are deployed onto the edge nodes,the users’ inference tasks need to be unloaded to the edge nodes for performing,therefore a distributed decision-making mechanism for task offloading is proposed in this dissertation.The main research work is as follows:This dissertation proposes a semi-asynchronous federated learning(FedSA)mechanism to implement efficient distributed model training in edge computing,taking into account the edge heterogeneity and resource constraints.Specifically,in each round,after the parameter server receives the local models from a certain number(e.g.,M)of workers,it aggregates those local models,depending on workers’ arrival order at the parameter server.Furthermore,this dissertation proves the convergence of FedSA and analyzes the quantitative relationship between training performance and several factors,such as the number M of workers participating in global updating,data distribution,edge heterogeneity and communication budget.Based on this,this dissertation proposes an efficient algorithm to determine the optimal value of M according to edge heterogeneity and data distribution among workers,so as to minimize the training time given the communication budget.In addition,this dissertation deploys adaptive learning rate for edge nodes according to their relative frequency participating in global update,which further improves the training accuracy under Non-IID data.The proposed mechanism and algorithm are implemented on the testbed,and the experimental results show that the performance of the proposed mechanism is better than that of the existing mechanism on the data sets with different degrees of Non-IID.This dissertation proposes a unified hierarchical federated learning clustering(FedUC)mechanism to accelerate distributed model training,taking into account the edge heterogeneity,resource constraints and numerous devices.This dissertation explores the quantitative relationship between the convergence bounds of three different intercluster patterns(i.e.,CenSyn,CenAsy and DecSyn)and several factors,e.g.,data distribution among clusters,frequency of clusters participating in inter-cluster aggregation,and inter-cluster topology.For intra-cluster aggregation,this dissertation proposes a time-sharing scheduling strategy,called magic mirror method(TS-3M)and prove that TS-3M can minimize the completion time of intra-cluster aggregation.Based on both the convergence analysis for inter-cluster aggregation patterns and the optimal intracluster aggregation strategy,this dissertation designs a unified clustering algorithm FedUC to solve the cluster construction problem for hierarchical federated learning.FedUC can provide the optimal clustering strategies for hierarchical federated learning under different intra-cluster aggregation patterns.Experimental results show that,by deploying our cluster construction algorithm,this dissertation can greatly accelerate the model training of CenSyn(2.67×),CenAsy(3.03×)and DecSyn(8.32×).This dissertation proposes a distributed task offloading decision-making(DTO)mechanism to implement efficient distributed knowledge inference in edge computing,taking into account the edge heterogeneity,resource constraints,numerous devices and environmental dynamics.In DTO,users make their offloading decisions independently only based on the local information(e.g.,the current transmission channel,resource usage of neighboring edge servers).Since the decision making process takes a quite short time,and the decision making time will not increase with the increasing scale of network.Thus,it has better scalability and can provide real-time offloading decisions for users in high dynamic scenarios.There are two optional algorithms on the user side.One is the greedy-based algorithm DTO-UG,with very simple computation and short decision-making time,and the other is called DTO-UX based on convex optimization,which has better decision quality.This dissertation also theoretically proves that the proposed algorithms will converge to the approximate global optimal.This dissertation sets up a testbed and implements a latency-sensitive application,Vehicle Detection,to analyze the effectiveness of the offloading algorithms.This dissertation also conducts large-scale simulations for numerical evaluation.The results of simulations and system implementation on the testbed show that DTO can reduce the average task response time by about 50%-65%compared with the existing algorithms while keeping a short decision making time. |
