Implicit neural representations store videos as neural networks and have performed well for various vision tasks such as video compression and denoising. With frame index or positional index as input, implicit representations (NeRV, E-NeRV, \etc) reconstruct video from fixed and content-agnostic embeddings. Such embedding largely limits the regression capacity and internal generalization for video interpolation. In this paper, we propose a Hybrid Neural Representation for Videos (HNeRV), where a learnable encoder generates content-adaptive embeddings, which act as the decoder input. Besides the input embedding, we introduce HNeRV blocks, which ensure model parameters are evenly distributed across the entire network, such that higher layers (layers near the output) can have more capacity to store high-resolution content and video details. With content-adaptive embeddings and re-designed architecture, HNeRV outperforms implicit methods in video regression tasks for both reconstruction quality ($+4.7$ PSNR) and convergence speed ($16\times$ faster), and shows better internal generalization. As a simple and efficient video representation, HNeRV also shows decoding advantages for speed, flexibility, and deployment, compared to traditional codecs~(H.264, H.265) and learning-based compression methods. Finally, we explore the effectiveness of HNeRV on downstream tasks such as video compression and video inpainting. We provide project page at https://haochen-rye.github.io/HNeRV, and Code at https://github.com/haochen-rye/HNeRV

翻译：隐式神经表示将视频存储为神经网络，并在视频压缩、去噪等视觉任务中表现出色。以帧索引或位置索引为输入时，隐式表示（如NeRV、E-NeRV等）通过固定且内容无关的嵌入重建视频。此类嵌入极大限制了视频插值任务中的回归能力与内部泛化性能。本文提出一种混合神经视频表示方法（HNeRV），其中可学习的编码器生成内容自适应嵌入，作为解码器的输入。除输入嵌入外，我们引入HNeRV模块，确保模型参数均匀分布在网络各处，使高层（靠近输出的层）具备更强能力存储高分辨率内容与视频细节。凭借内容自适应嵌入与重新设计的架构，HNeRV在视频回归任务中重建质量（+4.7 PSNR）与收敛速度（快16倍）均优于隐式方法，并展现出更优的内部泛化能力。作为一种简单高效的视频表示方法，HNeRV在速度、灵活性和部署方面相比传统编解码器（H.264、H.265）及基于学习的压缩方法亦具解码优势。最后，我们探索HNeRV在视频压缩与视频修复等下游任务中的有效性。项目页面与代码分别发布在https://haochen-rye.github.io/HNeRV及https://github.com/haochen-rye/HNeRV。