While recent machine learning research has revealed connections between deep generative models such as VAEs and rate-distortion losses used in learned compression, most of this work has focused on images. In a similar spirit, we view recently proposed neural video coding algorithms through the lens of deep autoregressive and latent variable modeling. We present these codecs as instances of a generalized stochastic temporal autoregressive transform, and propose new avenues for further improvements inspired by normalizing flows and structured priors. We propose several architectures that yield state-of-the-art video compression performance on high-resolution video and discuss their tradeoffs and ablations. In particular, we propose (i) improved temporal autoregressive transforms, (ii) improved entropy models with structured and temporal dependencies, and (iii) variable bitrate versions of our algorithms. Since our improvements are compatible with a large class of existing models, we provide further evidence that the generative modeling viewpoint can advance the neural video coding field.

翻译：尽管近期机器学习研究揭示了深度生成模型（如变分自编码器）与学习压缩中的率失真损失之间的关联，但大多数工作聚焦于图像。受此启发，我们从深度自回归和潜变量建模的角度审视新近提出的神经视频编码算法。我们将这些编解码器视为广义随机时序自回归变换的实例，并提出基于归一化流和结构化先验的新改进路径。我们提出了若干架构，在高分辨率视频上实现了最先进的视频压缩性能，并讨论了其权衡与消融实验。具体而言，我们提出：(i)改进的时序自回归变换，(ii)具有结构化与时序依赖性的改进熵模型，以及(iii)算法的变比特率变体。由于我们的改进与大量现有模型兼容，这进一步证明了生成式建模视角能够推动神经视频编码领域的发展。