We address end-to-end learned video compression with a special focus on better learning and utilizing temporal contexts. For temporal context mining, we propose to store not only the previously reconstructed frames, but also the propagated features into the generalized decoded picture buffer. From the stored propagated features, we propose to learn multi-scale temporal contexts, and re-fill the learned temporal contexts into the modules of our compression scheme, including the contextual encoder-decoder, the frame generator, and the temporal context encoder. Our scheme discards the parallelization-unfriendly auto-regressive entropy model to pursue a more practical decoding time. We compare our scheme with x264 and x265 (representing industrial software for H.264 and H.265, respectively) as well as the official reference software for H.264, H.265, and H.266 (JM, HM, and VTM, respectively). When intra period is 32 and oriented to PSNR, our scheme outperforms H.265--HM by 14.4% bit rate saving; when oriented to MS-SSIM, our scheme outperforms H.266--VTM by 21.1% bit rate saving.

翻译：我们针对端到端学习型视频压缩展开研究，重点聚焦于时间上下文的更优学习与利用。在时间上下文挖掘方面，我们提出不仅存储先前重建帧，还将传播特征存储至广义解码图像缓冲区。基于存储的传播特征，我们提出学习多尺度时间上下文，并将所学时间上下文重新填充至压缩方案的各个模块，包括上下文编码器-解码器、帧生成器及时间上下文编码器。本方案摒弃了不利于并行化的自回归熵模型，以追求更实用的解码时间。我们将本方案与x264、x265（分别代表H.264与H.265的工业级软件）以及H.264、H.265、H.266的官方参考软件（对应JM、HM、VTM）进行对比。当帧内周期设为32且以PSNR为优化目标时，本方案相较于H.265—HM节省14.4%的码率；当以MS-SSIM为优化目标时，本方案相较于H.266—VTM节省21.1%的码率。