Most existing image compression approaches perform transform coding in the pixel space to reduce its spatial redundancy. However, they encounter difficulties in achieving both high-realism and high-fidelity at low bitrate, as the pixel-space distortion may not align with human perception. To address this issue, we introduce a Generative Latent Coding (GLC) architecture, which performs transform coding in the latent space of a generative vector-quantized variational auto-encoder (VQ-VAE), instead of in the pixel space. The generative latent space is characterized by greater sparsity, richer semantic and better alignment with human perception, rendering it advantageous for achieving high-realism and high-fidelity compression. Additionally, we introduce a categorical hyper module to reduce the bit cost of hyper-information, and a code-prediction-based supervision to enhance the semantic consistency. Experiments demonstrate that our GLC maintains high visual quality with less than 0.04 bpp on natural images and less than 0.01 bpp on facial images. On the CLIC2020 test set, we achieve the same FID as MS-ILLM with 45% fewer bits. Furthermore, the powerful generative latent space enables various applications built on our GLC pipeline, such as image restoration and style transfer. The code is available at https://github.com/jzyustc/GLC.

翻译：大多数现有图像压缩方法在像素空间进行变换编码以减少其空间冗余。然而，在低码率下同时实现高真实感与高保真度面临困难，因为像素空间的失真可能与人眼感知不一致。为解决此问题，我们提出了一种生成式潜在编码架构，该架构在生成式向量量化变分自编码器的潜在空间而非像素空间执行变换编码。生成式潜在空间具有更高的稀疏性、更丰富的语义以及更好的人眼感知对齐特性，这使其有利于实现高真实感与高保真的压缩。此外，我们引入了类别化超信息模块以降低超信息的码率成本，并采用基于码字预测的监督机制以增强语义一致性。实验表明，我们的方法在自然图像上以低于0.04 bpp、在人脸图像上以低于0.01 bpp的码率仍能保持高视觉质量。在CLIC2020测试集上，我们以比MS-ILLM少45%的码率达到了相同的FID指标。此外，强大的生成式潜在空间使得基于我们压缩框架的多种应用成为可能，如图像修复与风格迁移。代码已开源：https://github.com/jzyustc/GLC。