| Machine learning(ML)systems,as the most important foundational software systems for artificial intelligence applications,need to address the challenge of massive privacy data leaks caused by the explosive growth of data.At the same time,the increasingly common use of artificial intelligence applications in mobile scenarios imposes more stringent requirements on the performance indicators of ML systems,such as computation,network,and privacy.Federated learning(FL)is a distributed ML approach that aims to train ML models without the need to transfer raw data from participants(such as mobile devices,edge devices,or data centers)to a central server.Among them,large-scale FL systems for mobile devices distribute the computation of model training and inference across a massive number of dispersed mobile devices.Under the unified coordination of cloud servers,these systems enable the sharing of knowledge learned by individual device models across networks.This presents new opportunities for building high-performance privacy-preserving ML systems.However,FL systems designed for mobile devices face significant challenges due to terminal resource limitations.Three key challenges can be distilled from the aspects of terminal,network,and cloud in federated learning:limited processing capabilities of mobile devices make it difficult to support complex deep neural network model training tasks;the wireless network environment of mobile devices creates severe communication bottlenecks for federated learning;and the stringent constraints of differential privacy result in a reduced throughput for federated learning systems.Therefore,this paper rethinks the research on efficient optimization methods for federated learning systems tailored to resource-constrained devices,and conducts research at three levels:terminal-side computation,network transmission,and privacy resource scheduling.·To address the mismatch between the low computational power of mobile devices and the high workload of model training,research is conducted on low-precision federated training techniques using heterogeneous chip cooperative computing.An FL framework based on low-precision operators is proposed,and an error compensation aggregation protocol is designed to aggregate model updates generated from local low-precision training instead of locally updating model weights.A pipeline batch quantization mechanism is designed to parallelize 32-bit float-based quantization and 8-bit integer-based training,significantly reducing the energy consumption of mobile devices and accelerating model convergence.This approach leverages the combination of algorithms and hardware,introducing new heterogeneous computing chips to enhance the computational power of mobile devices.Additionally,it designs low-precision training techniques based on the integer-friendly characteristics of these chips.·To tackle the issues of unstable wireless networks and slow convergence of federated models,research is conducted on unreliable and efficient transmission protocols based on the loss-tolerant characteristics of ML.A frozen training framework for the federated layer is proposed,along with a new high-throughput transmission protocol that tolerates partial parameter loss to achieve high-throughput transmission of data packets at the granularity of layers,which aims to accelerate FL training,and reduce communication costs.This method is inspired by the parameter loss-tolerant nature of ML.It guides the design of network layer transmission protocols with prior knowledge from ML.Taking into account that federated learning can tolerate partial parameter loss,it designs a federated frozen training framework to accelerate parameter transmission.·To address the trade-off between strong privacy guarantees of differential privacy and high throughput of federated systems,research is conducted on efficient allocation of privacy resources under task service level objective constraints.A unified differentially private FL platform is proposed,which introduces the concept of devicetime blocks,dividing the continuous data flow into fixed time frames on each device.Furthermore,a novel fine-grained privacy budget scheduling algorithm is introduced.This algorithm maximizes the number of completed tasks while reducing the violation rate of service-level objectives under limited privacy budget constraints.This method introduces the concept that "privacy is also a resource" and utilizes this model to design a unified differential privacy federated learning platform.It treats privacy budget as a system resource on par with CPU and memory,leading to the development of efficient privacy resource scheduling algorithms. |
