Recently, learned image compression has achieved remarkable performance. The entropy model, which estimates the distribution of the latent representation, plays a crucial role in boosting rate-distortion performance. However, most entropy models only capture correlations in one dimension, while the latent representation contain channel-wise, local spatial, and global spatial correlations. To tackle this issue, we propose the Multi-Reference Entropy Model (MEM) and the advanced version, MEM$^+$. These models capture the different types of correlations present in latent representation. Specifically, We first divide the latent representation into slices. When decoding the current slice, we use previously decoded slices as context and employ the attention map of the previously decoded slice to predict global correlations in the current slice. To capture local contexts, we introduce two enhanced checkerboard context capturing techniques that avoids performance degradation. Based on MEM and MEM$^+$, we propose image compression models MLIC and MLIC$^+$. Extensive experimental evaluations demonstrate that our MLIC and MLIC$^+$ models achieve state-of-the-art performance, reducing BD-rate by $8.05\%$ and $11.39\%$ on the Kodak dataset compared to VTM-17.0 when measured in PSNR. Our code will be available at https://github.com/JiangWeibeta/MLIC.
翻译:近年来,学习型图像压缩取得了显著性能。熵模型通过估计潜在表示的分布,在提升率失真性能中发挥关键作用。然而,现有熵模型大多仅捕捉单一维度的相关性,而潜在表示包含通道间、局部空间和全局空间三种相关性。为解决该问题,我们提出多参考熵模型(MEM)及其进阶版本MEM$^+$。这些模型能够捕捉潜在表示中不同类型相关性。具体而言,我们首先将潜在表示划分为多个切片。在解码当前切片时,利用先前已解码切片作为上下文,并通过先前解码切片的注意力图预测当前切片的全局相关性。为捕捉局部上下文,我们引入两种增强型棋盘格上下文捕捉技术,以避免性能退化。基于MEM与MEM$^+$,我们进一步提出图像压缩模型MLIC与MLIC$^+$。大量实验评估表明,我们的MLIC与MLIC$^+$模型实现了最先进性能,在Kodak数据集上以PSNR为指标相比VTM-17.0分别降低BD-rate $8.05\%$和$11.39\%$。相关代码将发布于https://github.com/JiangWeibeta/MLIC。