This paper introduces an online motion rate adaptation scheme for learned video compression, with the aim of achieving content-adaptive coding on individual test sequences to mitigate the domain gap between training and test data. It features a patch-level bit allocation map, termed the $\alpha$-map, to trade off between the bit rates for motion and inter-frame coding in a spatially-adaptive manner. We optimize the $\alpha$-map through an online back-propagation scheme at inference time. Moreover, we incorporate a look-ahead mechanism to consider its impact on future frames. Extensive experimental results confirm that the proposed scheme, when integrated into a conditional learned video codec, is able to adapt motion bit rate effectively, showing much improved rate-distortion performance particularly on test sequences with complicated motion characteristics.
翻译:本文提出了一种面向学习型视频压缩的在线运动码率自适应方法,旨在通过对单个测试序列实现内容自适应编码来缓解训练数据与测试数据之间的域差异。该方法引入了一种基于图像块的比特分配图(称为α-map),以空间自适应方式在运动编码和帧间编码的码率之间进行权衡。我们在推理阶段通过在线反向传播机制优化α-map,并进一步结合前向预测机制以考虑其对未来帧的影响。大量实验结果表明,将所提方案集成到条件学习型视频编码器中后,能够有效调节运动码率,尤其在具有复杂运动特性的测试序列上展现出显著提升的率失真性能。