As Internet of Things (IoT) technology advances, end devices like sensors and smartphones are progressively equipped with AI models tailored to their local memory and computational constraints. Local inference reduces communication costs and latency; however, these smaller models typically underperform compared to more sophisticated models deployed on edge servers or in the cloud. Cooperative Inference Systems (CISs) address this performance trade-off by enabling smaller devices to offload part of their inference tasks to more capable devices. These systems often deploy hierarchical models that share numerous parameters, exemplified by Deep Neural Networks (DNNs) that utilize strategies like early exits or ordered dropout. In such instances, Federated Learning (FL) may be employed to jointly train the models within a CIS. Yet, traditional training methods have overlooked the operational dynamics of CISs during inference, particularly the potential high heterogeneity in serving rates across clients. To address this gap, we propose a novel FL approach designed explicitly for use in CISs that accounts for these variations in serving rates. Our framework not only offers rigorous theoretical guarantees, but also surpasses state-of-the-art (SOTA) training algorithms for CISs, especially in scenarios where inference request rates or data availability are uneven among clients.

翻译：随着物联网技术的发展，传感器和智能手机等终端设备逐渐配备适应其本地内存和计算约束的人工智能模型。本地推理能够降低通信成本和延迟，但与部署在边缘服务器或云端的复杂模型相比，这些小型模型通常性能较差。协作推理系统通过允许小型设备将部分推理任务卸载至性能更强的设备来解决这一性能权衡问题。这类系统常部署共享大量参数的分层模型，例如采用早退策略或有序丢弃机制的深度神经网络。在此类场景中，联邦学习可用于联合训练协作推理系统内的模型。然而，传统训练方法忽略了协作推理系统在推理过程中的运行动态，特别是客户端之间服务速率的潜在高度异质性。为解决这一问题，我们提出一种专为协作推理系统设计的新型联邦学习方法，该方法充分考虑服务速率的变化。我们的框架不仅提供了严格的理论保证，而且在推理请求速率或数据可用性不均衡的场景中，显著优于现有最先进的协作推理系统训练算法。