Neural video codecs have demonstrated great potential in video transmission and storage applications. Existing neural hybrid video coding approaches rely on optical flow or Gaussian-scale flow for prediction, which cannot support fine-grained adaptation to diverse motion content. Towards more content-adaptive prediction, we propose a novel cross-scale prediction module that achieves more effective motion compensation. Specifically, on the one hand, we produce a reference feature pyramid as prediction sources and then transmit cross-scale flows that leverage the feature scale to control the precision of prediction. On the other hand, for the first time, a weighted prediction mechanism is introduced even if only a single reference frame is available, which can help synthesize a fine prediction result by transmitting cross-scale weight maps. In addition to the cross-scale prediction module, we further propose a multi-stage quantization strategy, which improves the rate-distortion performance with no extra computational penalty during inference. We show the encouraging performance of our efficient neural video codec (ENVC) on several benchmark datasets. In particular, the proposed ENVC can compete with the latest coding standard H.266/VVC in terms of sRGB PSNR on UVG dataset for the low-latency mode. We also analyze in detail the effectiveness of the cross-scale prediction module in handling various video content, and provide a comprehensive ablation study to analyze those important components. Test code is available at https://github.com/USTC-IMCL/ENVC .

翻译：神经视频编解码器在视频传输与存储应用中展现出巨大潜力。现有神经混合视频编码方法依赖光流或高斯尺度流进行预测，无法实现对多样化运动内容的细粒度自适应。为提升内容自适应预测能力，本文提出一种新颖的跨尺度预测模块，可实现更高效的运动补偿。具体而言：一方面，我们构建参考特征金字塔作为预测源，并通过传输跨尺度流利用特征尺度控制预测精度；另一方面，首次引入加权预测机制（即使仅有单参考帧可用），通过传输跨尺度权重图合成精细预测结果。除跨尺度预测模块外，我们还提出多阶段量化策略，在推理阶段不增加额外计算代价的前提下提升率失真性能。我们在多个基准数据集上展示了所提出的高效神经视频编解码器（ENVC）的优异性能。特别地，在UVG数据集低延迟模式下，ENVC的sRGB PSNR指标可媲美最新编码标准H.266/VVC。此外，我们详细分析了跨尺度预测模块处理不同视频内容的有效性，并通过全面的消融实验剖析各重要组件的贡献。测试代码开源于：https://github.com/USTC-IMCL/ENVC。