Effective Adaptive BitRate (ABR) algorithm or policy is of paramount importance for Real-Time Video Communication (RTVC) amid this pandemic to pursue uncompromised quality of experience (QoE). Existing ABR methods mainly separate the network bandwidth estimation and video encoder control, and fine-tune video bitrate towards estimated bandwidth, assuming the maximization of bandwidth utilization yields the optimal QoE. However, the QoE of a RTVC system is jointly determined by the quality of compressed video, fluency of video playback, and interaction delay. Solely maximizing the bandwidth utilization without comprehensively considering compound impacts incurred by both network and video application layers, does not assure the satisfactory QoE. And the decoupling of network and video layer further exacerbates the user experience due to network-codec incoordination. This work therefore proposes the Palette, a reinforcement learning based ABR scheme that unifies the processing of network and video application layers to directly maximize the QoE formulated as the weighted function of video quality, stalling rate and delay. To this aim, a cross-layer optimization is proposed to derive fine-grained compression factor of upcoming frame(s) using cross-layer observations like network conditions, video encoding parameters, and video content complexity. As a result, Palette manages to resolve the network-codec incoordination and to best catch up with the network fluctuation. Compared with state-of-the-art schemes in real-world tests, Palette not only reduces 3.1\%-46.3\% of the stalling rate, 20.2\%-50.8\% of the delay, but also improves 0.2\%-7.2\% of the video quality with comparable bandwidth consumption, under a variety of application scenarios.

翻译：有效自适应比特率（ABR）算法或策略对于疫情期间追求无损体验质量（QoE）的实时视频通信（RTVC）至关重要。现有ABR方法主要将网络带宽估计与视频编码器控制分离，并针对估计带宽微调视频比特率，假设最大化带宽利用率即可获得最优QoE。然而，RTVC系统的QoE由压缩视频质量、视频播放流畅度和交互延迟共同决定。仅最大化带宽利用率而未全面考虑网络层与视频应用层产生的复合影响，无法保证满意的QoE。而网络层与视频层的解耦会因网络-编解码器不协调进一步恶化用户体验。为此，本文提出Palette——一种基于强化学习的ABR方案，该方案统一处理网络层与视频应用层，直接最大化由视频质量、卡顿率和延迟加权函数定义的QoE。为实现此目标，提出跨层优化方法，利用网络状况、视频编码参数和视频内容复杂度等跨层观测值，对后续帧推导细粒度压缩因子。最终，Palette成功解决了网络-编解码器不协调问题，并最佳地适应网络波动。在实际测试中，与现有先进方案相比，Palette在多种应用场景下不仅将卡顿率降低3.1%-46.3%、延迟降低20.2%-50.8%，还将视频质量提升0.2%-7.2%，同时保持相当的带宽消耗。