Active Queue Management (AQM) aims to prevent bufferbloat and serial drops in router and switch FIFO packet buffers that usually employ drop-tail queueing. AQM describes methods to send proactive feedback to TCP flow sources to regulate their rate using selective packet drops or markings. Traditionally, AQM policies relied on heuristics to approximately provide Quality of Service (QoS) such as a target delay for a given flow. These heuristics are usually based on simple network and TCP control models together with the monitored buffer filling. A primary drawback of these heuristics is that their way of accounting flow characteristics into the feedback mechanism and the corresponding effect on the state of congestion are not well understood. In this work, we show that taking a probabilistic model for the flow rates and the dequeueing pattern, a Semi-Markov Decision Process (SMDP) can be formulated to obtain an optimal packet-dropping policy. This policy-based AQM, named PAQMAN, takes into account a steady-state model of TCP and a target delay for the flows. Additionally, we present an inference algorithm that builds on TCP congestion control in order to calibrate the model parameters governing underlying network conditions. Using simulation, we show that the prescribed AQM yields comparable throughput to state-of-the-art AQM algorithms while reducing delays significantly.

翻译：主动队列管理（AQM）旨在防止通常采用尾部丢弃队列的路由器和交换机FIFO数据包缓冲区出现缓冲膨胀与串行丢包现象。AQM描述了通过选择性丢包或标记向TCP流源发送主动反馈以调节其速率的方法。传统上，AQM策略依赖启发式方法近似提供服务质量（QoS），例如针对特定流的目标延迟。这些启发式方法通常基于简单的网络和TCP控制模型以及监测的缓冲区填充状态。此类方法的主要缺陷在于：将流特征纳入反馈机制的方式及其对拥塞状态的相应影响尚未得到充分理解。在本工作中，我们证明通过建立流速率与出队模式的概率模型，可以构造半马尔可夫决策过程（SMDP）以获得最优丢包策略。这种基于策略的AQM（命名为PAQMAN）兼顾了TCP稳态模型与流的目标延迟。此外，我们提出一种基于TCP拥塞控制的推断算法，用于校准描述底层网络条件的模型参数。仿真结果表明，所提出的AQM在保持与现有先进AQM算法相当吞吐量的同时，显著降低了延迟。