Spectral super-resolution that aims to recover hyperspectral image (HSI) from easily obtainable RGB image has drawn increasing interest in the field of computational photography. The crucial aspect of spectral super-resolution lies in exploiting the correlation within HSIs. However, two types of bottlenecks in existing Transformers limit performance improvement and practical applications. First, existing Transformers often separately emphasize either spatial-wise or spectral-wise correlation, disrupting the 3D features of HSI and hindering the exploitation of unified spatial-spectral correlation. Second, existing self-attention mechanism always establishes full-rank correlation matrix by learning the correlation between pairs of tokens, leading to its inability to describe linear dependence widely existing in HSI among multiple tokens. To address these issues, we propose a novel Exhaustive Correlation Transformer (ECT) for spectral super-resolution. First, we propose a Spectral-wise Discontinuous 3D (SD3D) splitting strategy, which models unified spatial-spectral correlation by integrating spatial-wise continuous splitting strategy and spectral-wise discontinuous splitting strategy. Second, we propose a Dynamic Low-Rank Mapping (DLRM) model, which captures linear dependence among multiple tokens through a dynamically calculated low-rank dependence map. By integrating unified spatial-spectral attention and linear dependence, our ECT can model exhaustive correlation within HSI. The experimental results on both simulated and real data indicate that our method achieves state-of-the-art performance. Codes and pretrained models will be available later.

翻译：光谱超分辨率旨在从易获取的RGB图像中恢复高光谱图像（HSI），近年来在计算摄影领域引起了广泛关注。其关键挑战在于挖掘HSI内部的相关性。然而，现有Transformer模型中存在两类瓶颈，限制了性能提升与实际应用。第一，现有Transformer通常分别强调空间维或光谱维的相关性，破坏了HSI的三维特征，阻碍了统一空谱相关性的挖掘。第二，现有自注意力机制总是通过学习成对token之间的相关性来建立满秩相关性矩阵，导致其无法描述HSI中广泛存在的多token线性依赖关系。为解决这些问题，我们提出了一种新颖的穷尽相关性Transformer（ECT）用于光谱超分辨率。首先，我们提出了一种光谱维非连续三维（SD3D）分割策略，通过集成空间维连续分割策略与光谱维非连续分割策略来建模统一的空谱相关性。其次，我们提出了动态低秩映射（DLRM）模型，通过动态计算的低秩依赖图捕获多token之间的线性依赖关系。通过整合统一的空谱注意力与线性依赖，我们的ECT能够建模HSI内部的穷尽相关性。在模拟数据和真实数据上的实验结果表明，我们的方法达到了最先进的性能。代码与预训练模型稍后将公开。