This paper addresses the critical challenge of managing Quality of Service (QoS) in cloud services, focusing on the nuances of individual tenant expectations and varying Service Level Indicators (SLIs). It introduces a novel approach utilizing Deep Reinforcement Learning for tenant-specific QoS management in multi-tenant, multi-accelerator cloud environments. The chosen SLI, deadline hit rate, allows clients to tailor QoS for each service request. A novel online scheduling algorithm for Deep Neural Networks in multi-accelerator systems is proposed, with a focus on guaranteeing tenant-wise, model-specific QoS levels while considering real-time constraints.

翻译：本文探讨了云服务质量（QoS）管理中的关键挑战，重点关注单个租户期望的细微差别及不同的服务等级指标（SLI）。我们提出了一种新颖方法，利用深度强化学习在多租户、多加速器的云环境中实现租户特定的QoS管理。选取的SLI——截止时限命中率——使客户能够为每个服务请求定制QoS。本文提出了一种面向多加速器系统的深度神经网络在线调度算法，其核心是在考虑实时约束的前提下，保障租户级别、模型特定的QoS水平。