Many real-time applications (e.g., Augmented/Virtual Reality, cognitive assistance) rely on Deep Neural Networks (DNNs) to process inference tasks. Edge computing is considered a key infrastructure to deploy such applications, as moving computation close to the data sources enables us to meet stringent latency and throughput requirements. However, the constrained nature of edge networks poses several additional challenges to the management of inference workloads: edge clusters can not provide unlimited processing power to DNN models, and often a trade-off between network and processing time should be considered when it comes to end-to-end delay requirements. In this paper, we focus on the problem of scheduling inference queries on DNN models in edge networks at short timescales (i.e., few milliseconds). By means of simulations, we analyze several policies in the realistic network settings and workloads of a large ISP, highlighting the need for a dynamic scheduling policy that can adapt to network conditions and workloads. We therefore design ASET, a Reinforcement Learning based scheduling algorithm able to adapt its decisions according to the system conditions. Our results show that ASET effectively provides the best performance compared to static policies when scheduling over a distributed pool of edge resources.

翻译：许多实时应用（例如增强现实/虚拟现实、认知辅助）依赖于深度神经网络（DNN）来处理推理任务。边缘计算被视为部署此类应用的关键基础设施，因为将计算靠近数据源能够满足严格的延迟和吞吐量要求。然而，边缘网络的资源受限特性给推理工作负载的管理带来了额外挑战：边缘集群无法为DNN模型提供无限的处理能力，并且在端到端延迟要求方面，通常需要权衡网络传输时间与处理时间。本文关注于在边缘网络中，在短时间尺度（即数毫秒内）对DNN模型上的推理查询进行调度的问题。通过仿真，我们分析了大型互联网服务提供商（ISP）真实网络环境和工作负载下的多种策略，凸显了对能够适应网络条件和工作负载的动态调度策略的需求。因此，我们设计了ASET——一种基于强化学习的调度算法，能够根据系统状态自适应调整决策。结果表明，与静态策略相比，ASET在分布式边缘资源池上进行调度时能有效提供最优性能。