Multispectral Sentinel-2 images are a valuable source of Earth observation data, however spatial resolution of their spectral bands limited to 10 m, 20 m, and 60 m ground sampling distance remains insufficient in many cases. This problem can be addressed with super-resolution, aimed at reconstructing a high-resolution image from a low-resolution observation. For Sentinel-2, spectral information fusion allows for enhancing the 20 m and 60 m bands to the 10 m resolution. Also, there were attempts to combine multitemporal stacks of individual Sentinel-2 bands, however these two approaches have not been combined so far. In this paper, we introduce DeepSent -- a new deep network for super-resolving multitemporal series of multispectral Sentinel-2 images. It is underpinned with information fusion performed simultaneously in the spectral and temporal dimensions to generate an enlarged multispectral image. In our extensive experimental study, we demonstrate that our solution outperforms other state-of-the-art techniques that realize either multitemporal or multispectral data fusion. Furthermore, we show that the advantage of DeepSent results from how these two fusion types are combined in a single architecture, which is superior to performing such fusion in a sequential manner. Importantly, we have applied our method to super-resolve real-world Sentinel-2 images, enhancing the spatial resolution of all the spectral bands to 3.3 m nominal ground sampling distance, and we compare the outcome with very high-resolution WorldView-2 images. We will publish our implementation upon paper acceptance, and we expect it will increase the possibilities of exploiting super-resolved Sentinel-2 images in real-life applications.

翻译：多光谱哨兵-2图像是宝贵的地球观测数据源，但其光谱波段的空间分辨率受限于10米、20米和60米的地面采样距离，在许多情况下仍显不足。这一问题可通过超分辨率技术解决，该技术旨在从低分辨率观测中重建高分辨率图像。针对哨兵-2卫星，光谱信息融合可将20米和60米波段增强至10米分辨率。此外，已有研究尝试结合单个哨兵-2波段的多时相堆叠，但迄今为止这两种方法尚未被联合使用。本文提出DeepSent——一种用于多时相多光谱哨兵-2图像超分辨率的新型深度网络。该网络基于在光谱和时间维度上同步进行的信息融合，以生成放大的多光谱图像。通过广泛的实验研究，我们证明该方案优于实现多时相或多光谱数据融合的其他先进技术。进一步地，研究表明DeepSent的优势源于其单一架构中两种融合方式的组合方式，这优于顺序执行此类融合的方法。重要的是，我们将该方法应用于真实哨兵-2图像的超分辨率处理，将所有光谱波段的空间分辨率提升至3.3米名义地面采样距离，并将结果与极高分辨率WorldView-2图像进行对比。论文录用后，我们将公开代码，并期待其能提升超分辨率哨兵-2图像在实际应用中的利用可能性。