Learning-based video compression is currently a popular research topic, offering the potential to compete with conventional standard video codecs. In this context, Implicit Neural Representations (INRs) have previously been used to represent and compress image and video content, demonstrating relatively high decoding speed compared to other methods. However, existing INR-based methods have failed to deliver rate quality performance comparable with the state of the art in video compression. This is mainly due to the simplicity of the employed network architectures, which limit their representation capability. In this paper, we propose HiNeRV, an INR that combines light weight layers with novel hierarchical positional encodings. We employs depth-wise convolutional, MLP and interpolation layers to build the deep and wide network architecture with high capacity. HiNeRV is also a unified representation encoding videos in both frames and patches at the same time, which offers higher performance and flexibility than existing methods. We further build a video codec based on HiNeRV and a refined pipeline for training, pruning and quantization that can better preserve HiNeRV's performance during lossy model compression. The proposed method has been evaluated on both UVG and MCL-JCV datasets for video compression, demonstrating significant improvement over all existing INRs baselines and competitive performance when compared to learning-based codecs (72.3% overall bit rate saving over HNeRV and 43.4% over DCVC on the UVG dataset, measured in PSNR).

翻译：基于学习的视频压缩是当前的热门研究课题，有望与传统标准视频编解码器相竞争。在此背景下，隐式神经表示（INRs）此前已被用于表示和压缩图像与视频内容，并展现出相比其他方法较高的解码速度。然而，现有基于INR的方法未能提供与视频压缩领域最先进技术相媲美的率失真性能，这主要归因于所采用的网络架构过于简单，限制了其表征能力。本文提出HiNeRV，一种将轻量级层与新颖层级位置编码相结合的INR方法。我们采用深度可分离卷积、MLP和插值层构建具有高容量的深层宽网络架构。HiNeRV还是一种统一表示，可同时以帧和块形式编码视频，相比现有方法提供了更高的性能和灵活性。我们进一步构建了基于HiNeRV的视频编解码器，并优化了训练、剪枝和量化流程，以在有损模型压缩过程中更好地保持HiNeRV的性能。所提方法在UVG和MCL-JCV数据集上进行了视频压缩评估，结果表明其显著优于所有现有INR基线，并与基于学习的编解码器相比具有竞争力（在UVG数据集上以PSNR衡量，总体比特率节省较HNeRV达72.3%，较DCVC达43.4%）。