Machine learning models often fail to generalize well under distributional shifts. Understanding and overcoming these failures have led to a research field of Out-of-Distribution (OOD) generalization. Despite being extensively studied for static computer vision tasks, OOD generalization has been underexplored for time series tasks. To shine light on this gap, we present WOODS: eight challenging open-source time series benchmarks covering a diverse range of data modalities, such as videos, brain recordings, and sensor signals. We revise the existing OOD generalization algorithms for time series tasks and evaluate them using our systematic framework. Our experiments show a large room for improvement for empirical risk minimization and OOD generalization algorithms on our datasets, thus underscoring the new challenges posed by time series tasks. Code and documentation are available at https://woods-benchmarks.github.io .

翻译：机器学习模型在分布变化下通常难以良好泛化。理解并克服这些失败催生了一个名为分布外泛化的研究领域。尽管这一领域在静态计算机视觉任务中已被广泛研究，但在时间序列任务中的探索仍显不足。为填补这一空白，我们提出WOODS：八个具有挑战性的开源时间序列基准，涵盖多样化数据模态，如视频、脑电记录和传感器信号。我们改进了现有的时间序列任务分布外泛化算法，并利用系统化框架对其进行评估。实验表明，经验风险最小化与分布外泛化算法在数据集上仍有极大提升空间，从而凸显了时间序列任务所带来的新挑战。代码与文档见https://woods-benchmarks.github.io。