Our objective is to discover and localize monotonic temporal changes in a sequence of images. To achieve this, we exploit a simple proxy task of ordering a shuffled image sequence, with `time' serving as a supervisory signal since only changes that are monotonic with time can give rise to the correct ordering. We also introduce a flexible transformer-based model for general-purpose ordering of image sequences of arbitrary length with built-in attribution maps. After training, the model successfully discovers and localizes monotonic changes while ignoring cyclic and stochastic ones. We demonstrate applications of the model in multiple video settings covering different scene and object types, discovering both object-level and environmental changes in unseen sequences. We also demonstrate that the attention-based attribution maps function as effective prompts for segmenting the changing regions, and that the learned representations can be used for downstream applications. Finally, we show that the model achieves the state of the art on standard benchmarks for ordering a set of images.

翻译：我们的目标是发现并定位图像序列中的单调时序变化。为此，我们利用一个简单的代理任务——对打乱的图像序列进行排序，其中“时间”作为监督信号，因为只有与时间保持单调性的变化才能产生正确的排序。我们还引入了一种基于Transformer的灵活模型，用于对任意长度的图像序列进行通用排序，并内置注意力权重图。训练完成后，该模型能够成功发现并定位单调变化，同时忽略循环性和随机性变化。我们展示了该模型在多种视频场景（涵盖不同场景和对象类型）中的应用，能够在未见过的序列中发现对象级和环境级的变化。此外，我们还证明基于注意力的权重图可作为有效提示用于分割变化区域，且学习到的表征可用于下游应用。最后，我们展示了该模型在图像排序标准基准测试中达到了当前最优性能。