Hyperspectral imaging systems that use multispectral filter arrays (MSFA) capture only one spectral component in each pixel. Hyperspectral demosaicing is used to recover the non-measured components. While deep learning methods have shown promise in this area, they still suffer from several challenges, including limited modeling of non-local dependencies, lack of consideration of the periodic MSFA pattern that could be linked to periodic artifacts, and difficulty in recovering high-frequency details. To address these challenges, this paper proposes a novel de-mosaicing framework, the MSFA-frequency-aware Transformer network (FDM-Net). FDM-Net integrates a novel MSFA-frequency-aware multi-head self-attention mechanism (MaFormer) and a filter-based Fourier zero-padding method to reconstruct high pass components with greater difficulty and low pass components with relative ease, separately. The advantage of Maformer is that it can leverage the MSFA information and non-local dependencies present in the data. Additionally, we introduce a joint spatial and frequency loss to transfer MSFA information and enhance training on frequency components that are hard to recover. Our experimental results demonstrate that FDM-Net outperforms state-of-the-art methods with 6dB PSNR, and reconstructs high-fidelity details successfully.

翻译：采用多光谱滤波阵列（MSFA）的高光谱成像系统在每个像素中仅捕获一个光谱分量。高光谱去马赛克技术用于恢复未测量的分量。尽管深度学习方法在该领域展现出潜力，但仍面临若干挑战，包括非局部依赖关系的建模能力有限、未充分考虑可能导致周期性伪影的MSFA周期模式，以及难以恢复高频细节。为解决这些问题，本文提出了一种新颖的去马赛克框架——MSFA频率感知Transformer网络（FDM-Net）。FDM-Net集成了一种新型MSFA频率感知多头自注意力机制（MaFormer）和基于滤波器的傅里叶零填充方法，分别重建难度较大的高通分量和相对容易的低通分量。MaFormer的优势在于能够利用数据中的MSFA信息和非局部依赖关系。此外，我们引入了一种联合空间与频率损失函数，以传递MSFA信息并增强对难以恢复的频率分量的训练。实验结果表明，FDM-Net以6dB的峰值信噪比（PSNR）超越了现有最先进方法，并成功重建了高保真细节。