Learned B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. However, previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation or video frame interpolation. They suffer from inaccurate quantized motions and inefficient motion compensation. To address these issues, we propose a simple yet effective structure called Interpolation-driven B-frame Video Compression (IBVC). Our approach only involves two major operations: video frame interpolation and artifact reduction compression. IBVC introduces a bit-rate free MEMC based on interpolation, which avoids optical-flow quantization and additional compression distortions. Later, to reduce duplicate bit-rate consumption and focus on unaligned artifacts, a residual guided masking encoder is deployed to adaptively select the meaningful contexts with interpolated multi-scale dependencies. In addition, a conditional spatio-temporal decoder is proposed to eliminate location errors and artifacts instead of using MEMC coding in other methods. The experimental results on B-frame coding demonstrate that IBVC has significant improvements compared to the relevant state-of-the-art methods. Meanwhile, our approach can save bit rates compared with the random access (RA) configuration of H.266 (VTM). The code will be available at https://github.com/ruhig6/IBVC.
翻译:学习型B帧视频压缩旨在采用双向运动估计与运动补偿编码进行中间帧重建。然而,以往的学习方法通常将神经P帧编解码器直接扩展到B帧,依赖双向光流估计或视频帧插值,存在量化运动不精确与运动补偿效率低下的问题。针对这些挑战,我们提出一种简洁高效的结构——插值驱动的B帧视频压缩(IBVC)。该方法仅涉及两大核心操作:视频帧插值与伪影缩减压缩。IBVC引入基于插值的无码率运动补偿,避免了光流量化及额外压缩失真。为降低冗余码率消耗并聚焦于未对齐伪影,我们部署了残差引导掩码编码器,通过插值多尺度依赖关系自适应选择有意义上下文。此外,提出条件时空解码器以消除位置误差与伪影,替代其他方法中的运动补偿编码。B帧编码实验结果表明,IBVC相较于相关最优方法具有显著提升。同时,与H.266(VTM)的随机接入(RA)配置相比,本方法可节省码率。代码将发布于https://github.com/ruhig6/IBVC。