Implicit neural representations (INR) have gained increasing attention in representing 3D scenes and images, and have been recently applied to encode videos (e.g., NeRV, E-NeRV). While achieving promising results, existing INR-based methods are limited to encoding a handful of short videos (e.g., seven 5-second videos in the UVG dataset) with redundant visual content, leading to a model design that fits individual video frames independently and is not efficiently scalable to a large number of diverse videos. This paper focuses on developing neural representations for a more practical setup -- encoding long and/or a large number of videos with diverse visual content. We first show that instead of dividing videos into small subsets and encoding them with separate models, encoding long and diverse videos jointly with a unified model achieves better compression results. Based on this observation, we propose D-NeRV, a novel neural representation framework designed to encode diverse videos by (i) decoupling clip-specific visual content from motion information, (ii) introducing temporal reasoning into the implicit neural network, and (iii) employing the task-oriented flow as intermediate output to reduce spatial redundancies. Our new model largely surpasses NeRV and traditional video compression techniques on UCF101 and UVG datasets on the video compression task. Moreover, when used as an efficient data-loader, D-NeRV achieves 3%-10% higher accuracy than NeRV on action recognition tasks on the UCF101 dataset under the same compression ratios.

翻译：隐式神经表征（INR）在三维场景和图像表示领域日益受到关注，并最近被应用于视频编码（如NeRV、E-NeRV）。尽管取得了令人瞩目的成果，现有基于INR的方法仅能编码少量内容冗余的短视频（例如 UVG 数据集中的七个5秒视频），其模型设计独立拟合单个视频帧，无法高效扩展到大量多样视频。本文致力于为更实际的场景开发神经表示——编码包含丰富视觉内容的长视频和/或大量视频。我们首先证明：与将视频分割为小子集并用独立模型编码相比，用统一模型联合编码长视频和多样视频可获得更优的压缩效果。基于这一发现，我们提出D-NeRV——一种专为编码多样视频设计的新型神经表示框架，其核心创新包括：（i）解耦片段特定视觉内容与运动信息，（ii）在隐式神经网络中引入时序推理能力，以及（iii）采用任务导向光流作为中间输出以减少空间冗余。在UCF101和UVG数据集上的视频压缩任务中，我们的新模型大幅超越NeRV及传统视频压缩技术。此外，当作为高效数据加载器时，在相同压缩比下，D-NeRV在UCF101数据集的动作识别任务上比NeRV的准确率提升3%-10%。