Neural video compression (NVC) is a rapidly evolving video coding research area, with some models achieving superior coding efficiency compared to the latest video coding standard Versatile Video Coding (VVC). In conventional video coding standards, the hierarchical B-frame coding, which utilizes a bidirectional prediction structure for higher compression, had been well-studied and exploited. In NVC, however, limited research has investigated the hierarchical B scheme. In this paper, we propose an NVC model exploiting hierarchical B-frame coding with temporal layer-adaptive optimization. We first extend an existing unidirectional NVC model to a bidirectional model, which achieves -21.13% BD-rate gain over the unidirectional baseline model. However, this model faces challenges when applied to sequences with complex or large motions, leading to performance degradation. To address this, we introduce temporal layer-adaptive optimization, incorporating methods such as temporal layer-adaptive quality scaling (TAQS) and temporal layer-adaptive latent scaling (TALS). The final model with the proposed methods achieves an impressive BD-rate gain of -39.86% against the baseline. It also resolves the challenges in sequences with large or complex motions with up to -49.13% more BD-rate gains than the simple bidirectional extension. This improvement is attributed to the allocation of more bits to lower temporal layers, thereby enhancing overall reconstruction quality with smaller bits. Since our method has little dependency on a specific NVC model architecture, it can serve as a general tool for extending unidirectional NVC models to the ones with hierarchical B-frame coding.

翻译：神经视频编码（NVC）是一个快速发展的视频编码研究领域，部分模型相比最新视频编码标准VVC（Versatile Video Coding）实现了更高的编码效率。在传统视频编码标准中，分层B帧编码利用双向预测结构实现更高压缩比，已得到充分研究和应用。然而，在NVC领域，针对分层B方案的研究仍较为有限。本文提出一种利用时间层级自适应优化的分层B帧编码的NVC模型。我们首先将现有单向NVC模型扩展为双向模型，该模型相比单向基线模型实现了-21.13%的BD-rate增益。但该模型在处理复杂或大幅运动序列时面临性能退化问题。为此，我们引入时间层级自适应优化方法，包括时间层级自适应质量缩放（TAQS）和时间层级自适应潜在缩放（TALS）。采用所提方法的最终模型相比基线实现了-39.86%的显著BD-rate增益，同时解决了大幅或复杂运动序列的挑战，相比简单双向扩展模型额外获得高达-49.13%的BD-rate增益。该改进归因于将更多比特分配给较低时间层级，从而以更少比特提升整体重建质量。由于本方法对具体NVC模型架构依赖度较低，可作为将单向NVC模型扩展为分层B帧编码模型的通用工具。