Real-time Internet of Things (IoT) applications require real-time support to handle the ever-growing demand for computing resources to process IoT workloads. Fog Computing provides high availability of such resources in a distributed manner. However, these resources must be efficiently managed to distribute unpredictable traffic demands among heterogeneous Fog resources. This paper proposes a fully distributed load-balancing solution with Multi-Agent Reinforcement Learning (MARL) that intelligently distributes IoT workloads to optimize the waiting time while providing fair resource utilization in the Fog network. These agents use transfer learning for life-long self-adaptation to dynamic changes in the environment. By leveraging distributed decision-making, MARL agents effectively minimize the waiting time compared to a single centralized agent solution and other baselines, enhancing end-to-end execution delay. Besides performance gain, a fully distributed solution allows for a global-scale implementation where agents can work independently in small collaboration regions, leveraging nearby local resources. Furthermore, we analyze the impact of a realistic frequency to observe the state of the environment, unlike the unrealistic common assumption in the literature of having observations readily available in real-time for every required action. The findings highlight the trade-off between realism and performance using an interval-based Gossip-based multi-casting protocol against assuming real-time observation availability for every generated workload.

翻译：实时物联网应用需要实时支持，以应对日益增长的算力需求来处理物联网工作负载。雾计算以分布式方式提供此类资源的高可用性，但这些资源必须得到高效管理，以便在异构雾资源间分配不可预测的流量负载。本文提出一种基于多智能体强化学习的全分布式负载均衡解决方案，该方案能够智能分配物联网工作负载，在优化等待时间的同时实现雾网络中的公平资源利用。这些智能体利用迁移学习实现对环境动态变化的终身自适应。通过分布式决策，多智能体强化学习智能体相较于单一集中式智能体方案及其他基线方法，有效最小化等待时间，提升端到端执行延迟。除性能提升外，全分布式方案支持全局规模部署，智能体可在小型协作区域内独立工作，充分利用附近本地资源。此外，我们分析了以实际频率观测环境状态的影响——区别于文献中普遍采用的非现实假设（即每次所需动作都能实时获取观测结果）。通过对比假设每次生成工作负载都能实时获取观测信息的情况，基于间隔的八卦多播协议揭示了真实性与性能之间的权衡关系。