Active Queue Management (AQM) is a mechanism employed to alleviate transient congestion in network device buffers, such as routers and switches. Traditional AQM algorithms use fixed thresholds, like target delay or queue occupancy, to compute random packet drop probabilities. A very small target delay can increase packet losses and reduce link utilization, while a large target delay may increase queueing delays while lowering drop probability. Due to dynamic network traffic characteristics, where traffic fluctuations can lead to significant queue variations, maintaining a fixed threshold AQM may not suit all applications. Consequently, we explore the question: \textit{What is the ideal threshold (target delay) for AQMs?} In this work, we introduce DESiRED (Dynamic, Enhanced, and Smart iRED), a P4-based AQM that leverages precise network feedback from In-band Network Telemetry (INT) to feed a Deep Reinforcement Learning (DRL) model. This model dynamically adjusts the target delay based on rewards that maximize application Quality of Service (QoS). We evaluate DESiRED in a realistic P4-based test environment running an MPEG-DASH service. Our findings demonstrate up to a 90x reduction in video stall and a 42x increase in high-resolution video playback quality when the target delay is adjusted dynamically by DESiRED.

翻译：主动队列管理（AQM）是一种用于缓解网络设备（如路由器和交换机）缓冲区瞬时拥塞的机制。传统AQM算法采用固定阈值（如目标延迟或队列占用率）来计算随机丢包概率。过小的目标延迟可能增加丢包率并降低链路利用率，而过大的目标延迟虽能降低丢包概率但可能增大排队延迟。由于网络流量具有动态特性，流量波动会导致队列显著变化，因此固定阈值的AQM可能无法适应所有应用场景。基于此，我们探讨了以下问题：*AQM的理想阈值（目标延迟）应如何确定？* 本文提出DESiRED（动态、增强与智能iRED），一种基于P4的AQM方案。该方法利用带内网络遥测（INT）提供的精准网络反馈，驱动深度强化学习（DRL）模型，根据最大化应用服务质量（QoS）的奖励机制动态调整目标延迟。我们在运行MPEG-DASH服务的真实P4测试环境中评估了DESiRED的性能。实验结果表明，当DESiRED动态调整目标延迟时，视频卡顿可减少90倍，高分辨率视频播放质量提升42倍。