Optical-flow-based and kernel-based approaches have been widely explored for temporal compensation in satellite video super-resolution (VSR). However, these techniques involve high computational consumption and are prone to fail under complex motions. In this paper, we proposed to exploit the well-defined temporal difference for efficient and robust temporal compensation. To fully utilize the temporal information within frames, we separately modeled the short-term and long-term temporal discrepancy since they provide distinctive complementary properties. Specifically, a short-term temporal difference module is designed to extract local motion representations from residual maps between adjacent frames, which provides more clues for accurate texture representation. Meanwhile, the global dependency in the entire frame sequence is explored via long-term difference learning. The differences between forward and backward segments are incorporated and activated to modulate the temporal feature, resulting in holistic global compensation. Besides, we further proposed a difference compensation unit to enrich the interaction between the spatial distribution of the target frame and compensated results, which helps maintain spatial consistency while refining the features to avoid misalignment. Extensive objective and subjective evaluation of five mainstream satellite videos demonstrates that the proposed method performs favorably for satellite VSR. Code will be available at \url{https://github.com/XY-boy/TDMVSR}
翻译:基于光流和基于核的方法已被广泛探索用于卫星视频超分辨率(VSR)中的时间补偿。然而,这些技术存在计算消耗高的问题,并且在复杂运动场景下容易失效。本文提出利用定义明确的时序差分实现高效且鲁棒的时间补偿。为充分利用帧间时间信息,我们分别建模短期和长期时间差异,因为它们能提供互补的独特特性。具体地,设计了一个短期时序差分模块,从相邻帧的残差图中提取局部运动表征,为精确纹理表示提供更多线索。同时,通过长期差分学习探索整个帧序列中的全局依赖关系。前向与后向片段之间的差异被整合并激活以调制时序特征,从而实现整体全局补偿。此外,我们进一步提出差分补偿单元,以丰富目标帧空间分布与补偿结果之间的交互,有助于在细化特征以避免错位的同时保持空间一致性。对五个主流卫星视频的主客观评估表明,所提方法在卫星VSR中表现优异。代码将在\url{https://github.com/XY-boy/TDMVSR}公布。