Self-supervised cross-modal super-resolution (SR) can overcome the difficulty of acquiring paired training data, but is challenging because only low-resolution (LR) source and high-resolution (HR) guide images from different modalities are available. Existing methods utilize pseudo or weak supervision in LR space and thus deliver results that are blurry or not faithful to the source modality. To address this issue, we present a mutual modulation SR (MMSR) model, which tackles the task by a mutual modulation strategy, including a source-to-guide modulation and a guide-to-source modulation. In these modulations, we develop cross-domain adaptive filters to fully exploit cross-modal spatial dependency and help induce the source to emulate the resolution of the guide and induce the guide to mimic the modality characteristics of the source. Moreover, we adopt a cycle consistency constraint to train MMSR in a fully self-supervised manner. Experiments on various tasks demonstrate the state-of-the-art performance of our MMSR.

翻译：为了解决这一问题,我们提出了一个相互调控的SR(MMSR)模型,通过相互调控战略,包括源对源的调制和源向导调导,来应对任务。在这些调制中,我们开发了跨部适应过滤器,以充分利用跨模式的空间依赖性,帮助引导源仿照指南的解析,引导导源模拟源模式特性。此外,我们采用了循环一致性限制,以完全自上而下的方式对MMSR进行培训。