The quality of experience (QoE) delivered by video conferencing systems to end users depends in part on correctly estimating the capacity of the bottleneck link between the sender and the receiver over time. Bandwidth estimation for real-time communications (RTC) remains a significant challenge, primarily due to the continuously evolving heterogeneous network architectures and technologies. From the first bandwidth estimation challenge which was hosted at ACM MMSys 2021, we learnt that bandwidth estimation models trained with reinforcement learning (RL) in simulations to maximize network-based reward functions may not be optimal in reality due to the sim-to-real gap and the difficulty of aligning network-based rewards with user-perceived QoE. This grand challenge aims to advance bandwidth estimation model design by aligning reward maximization with user-perceived QoE optimization using offline RL and a real-world dataset with objective rewards which have high correlations with subjective user-perceived audio/video quality in Microsoft Teams. All models submitted to the grand challenge underwent initial evaluation on our emulation platform. For a comprehensive evaluation under diverse network conditions with temporal fluctuations, top models were further evaluated on our geographically distributed testbed by using each model to conduct 600 calls within a 12-day period. The winning model is shown to deliver comparable performance to the top behavior policy in the released dataset. By leveraging real-world data and integrating objective audio/video quality scores as rewards, offline RL can therefore facilitate the development of competitive bandwidth estimators for RTC.

翻译：视频会议系统向最终用户提供的体验质量（QoE）部分取决于能否随时间准确估计发送方与接收方之间瓶颈链路的容量。实时通信（RTC）中的带宽估计仍是一项重大挑战，主要源于不断演进的异构网络架构与技术。从首届ACM MMSys 2021带宽估计挑战中，我们认识到：在仿真环境中使用强化学习（RL）训练以最大化基于网络的奖励函数的带宽估计模型，由于仿真与现实差距（sim-to-real gap）以及基于网络的奖励与用户感知QoE之间的对齐困难，在现实中可能并非最优。本项大规模挑战旨在通过以下方法推动带宽估计模型设计：利用离线强化学习，结合一个包含与Microsoft Teams中用户主观感知音视频质量高度相关的客观奖励的真实世界数据集，将奖励最大化与用户感知QoE优化对齐。所有提交至该挑战的模型均在我们的仿真平台上进行了初步评估。为在多种具有时间波动的网络条件下进行全面评估，顶尖模型进一步接受了为期12天的地理分布式测试床测试——每个模型执行600次通话。所胜出模型展现的性能与已发布数据集中的最优行为策略相当。因此，通过利用真实世界数据并将客观音视频质量评分作为奖励，离线强化学习能够促进开发具有竞争力的RTC带宽估计器。