Hyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes. We regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking. Quantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis. Our results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

翻译：高光谱成像若能在术中实时、高分辨率地实现组织表征，则具有改善术中决策的潜力。高光谱快照马赛克传感器因其采集速度快、体积紧凑而成为一种有前景的方法。然而，完全恢复快照图像的空间和光谱信息需要去马赛克算法。大多数最先进的去马赛克算法需要与高分辨率高光谱图像配对的真实标注训练数据，但在术中环境下，获取同一场景的精确配对图像物理上不可行。本研究提出一种完全无监督的高光谱图像去马赛克算法，该算法仅需示例快照图像进行训练。我们将高光谱去马赛克视为一个不适定的线性逆问题，并通过深度神经网络求解。利用自然场景中光谱相关性，我们设计了一种基于空间梯度一致性的新型光谱间正则化项。将所提正则化项与标准正则化技术结合，并利用标准数据保真项，构建了用于训练深度神经网络的无监督损失函数，从而实现实时高光谱图像去马赛克。高光谱图像数据集的定量结果表明，我们的无监督去马赛克方法可达到与监督方法相当的性能，且显著优于线性去马赛克方法。基于真实快照高光谱外科图像的定性用户研究证实了定量分析结果。我们的研究表明，所提出的无监督算法能实现有前景的实时高光谱去马赛克，从而提升该模态在术中应用的适用性。