In mobile edge computing (MEC), resource scheduling is crucial to task requests' performance and service providers' cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice. We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9.1x more profit than existing approaches without sacrificing the delay performance.

翻译：在移动边缘计算（MEC）中，资源调度对任务请求的性能和服务提供商的成本至关重要，涉及多层异构调度决策。现有调度器通常采用静态时间尺度来定期更新各层调度决策，未能针对不同层自适应调整时间尺度，导致实际性能可能欠佳。我们注意到，自适应时间尺度能显著改善运营成本与延迟性能之间的权衡。基于这一洞察，我们提出了EdgeTimer，这是首个利用深度强化学习（DRL）自动生成自适应时间尺度以更新多层调度决策的工作。首先，EdgeTimer采用三层分层DRL框架，将多层决策任务解耦为独立的子任务层次结构，以提高学习效率。其次，为应对每个子任务，EdgeTimer采用安全的多智能体DRL算法进行去中心化调度，同时确保系统可靠性。我们将EdgeTimer应用于广泛的Kubernetes调度规则，并使用具有不同工作负载模式的生产轨迹进行评估。大量轨迹驱动实验表明，无论工作负载模式和内置调度规则如何，EdgeTimer均能学习自适应时间尺度。在不牺牲延迟性能的前提下，其获得的收益可比现有方法高出多达9.1倍。