Content-adaptive compression is crucial for enhancing the adaptability of the pre-trained neural codec for various contents. Although these methods have been very practical in neural image compression (NIC), their application in neural video compression (NVC) is still limited due to two main aspects: 1), video compression relies heavily on temporal redundancy, therefore updating just one or a few frames can lead to significant errors accumulating over time; 2), NVC frameworks are generally more complex, with many large components that are not easy to update quickly during encoding. To address the previously mentioned challenges, we have developed a content-adaptive NVC technique called Group-aware Parameter-Efficient Updating (GPU). Initially, to minimize error accumulation, we adopt a group-aware approach for updating encoder parameters. This involves adopting a patch-based Group of Pictures (GoP) training strategy to segment a video into patch-based GoPs, which will be updated to facilitate a globally optimized domain-transferable solution. Subsequently, we introduce a parameter-efficient delta-tuning strategy, which is achieved by integrating several light-weight adapters into each coding component of the encoding process by both serial and parallel configuration. Such architecture-agnostic modules stimulate the components with large parameters, thereby reducing both the update cost and the encoding time. We incorporate our GPU into the latest NVC framework and conduct comprehensive experiments, whose results showcase outstanding video compression efficiency across four video benchmarks and adaptability of one medical image benchmark.
翻译:内容自适应压缩对于增强预训练神经编解码器对不同内容的适应性至关重要。尽管这些方法在神经图像压缩领域已非常实用,但它们在神经视频压缩中的应用仍受限于两个主要方面:1)视频压缩高度依赖时间冗余,因此仅更新一帧或少数几帧会导致误差随时间显著累积;2)神经视频压缩框架通常更为复杂,包含许多难以在编码过程中快速更新的大型组件。为解决上述挑战,我们提出了一种名为“组感知参数高效更新”的内容自适应神经视频压缩技术。首先,为最小化误差累积,我们采用组感知策略更新编码器参数:通过基于块的图像组训练策略将视频分割为基于块的GoP,并对这些分组进行更新以实现全局优化的域迁移解决方案。其次,我们引入参数高效的增量调优策略,通过将多个轻量级适配器以串行和并行配置集成到编码过程的每个编码组件中实现。这种与架构无关的模块能够激活具有大量参数的组件,从而降低更新成本与编码时间。我们将GPU技术融入最新神经视频压缩框架并开展全面实验,结果在四个视频基准测试中展现出卓越的视频压缩效率,并在一个医学图像基准测试中验证了其适应性。