The enhanced Deep Hierarchical Video Compression-DHVC 2.0-has been introduced. This single-model neural video codec operates across a broad range of bitrates, delivering not only superior compression performance to representative methods but also impressive complexity efficiency, enabling real-time processing with a significantly smaller memory footprint on standard GPUs. These remarkable advancements stem from the use of hierarchical predictive coding. Each video frame is uniformly transformed into multiscale representations through hierarchical variational autoencoders. For a specific scale's feature representation of a frame, its corresponding latent residual variables are generated by referencing lower-scale spatial features from the same frame and then conditionally entropy-encoded using a probabilistic model whose parameters are predicted using same-scale temporal reference from previous frames and lower-scale spatial reference of the current frame. This feature-space processing operates from the lowest to the highest scale of each frame, completely eliminating the need for the complexity-intensive motion estimation and compensation techniques that have been standard in video codecs for decades. The hierarchical approach facilitates parallel processing, accelerating both encoding and decoding, and supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss. Source codes will be made available.
翻译:增强型深度分层视频压缩模型——DHVC 2.0——已被提出。该单模型神经视频编解码器可在宽比特率范围内运行,不仅实现了优于代表性方法的压缩性能,还具备出色的计算效率,能够在标准GPU上以显著减少的内存占用实现实时处理。这些显著进展源于分层预测编码技术的运用。通过分层变分自编码器,每帧视频被统一转换为多尺度表征。针对特定尺度的帧特征表征,其对应的潜在残差变量通过参考同帧低尺度空间特征生成,随后使用概率模型进行条件熵编码——该概率模型的参数通过前帧同尺度时间参考与当前帧低尺度空间参考预测得到。此特征空间处理从每帧的最低尺度逐级执行至最高尺度,完全消除了传统视频编解码器数十年来依赖的高复杂度运动估计与补偿技术。分层架构促进了并行处理,加速了编码与解码过程,并支持传输友好的渐进式解码,使其在网络视频应用中面对数据包丢失时具有显著优势。源代码将公开提供。