Time Series Motif Discovery (TSMD), which aims at finding recurring patterns in time series, is an important task in numerous application domains, and many methods for this task exist. These methods are usually evaluated qualitatively. A few metrics for quantitative evaluation, where discovered motifs are compared to some ground truth, have been proposed, but they typically make implicit assumptions that limit their applicability. This paper introduces PROM, a broadly applicable metric that overcomes those limitations, and TSMD-Bench, a benchmark for quantitative evaluation of time series motif discovery. Experiments with PROM and TSMD-Bench show that PROM provides a more comprehensive evaluation than existing metrics, that TSMD-Bench is a more challenging benchmark than earlier ones, and that the combination can help understand the relative performance of TSMD methods. More generally, the proposed approach enables large-scale, systematic performance comparisons in this field.

翻译：时间序列模体发现（TSMD）旨在发现时间序列中的重复模式，是众多应用领域中的一项重要任务，目前存在多种方法。这些方法通常通过定性方式进行评估。已有少数定量评估指标被提出，通过将发现的模体与某些基准真值进行比较，但这些指标通常存在隐含假设，限制了其适用性。本文提出PROM——一种广泛适用且能克服这些局限性的评估指标，以及TSMD-Bench——一个用于时间序列模体发现定量评估的基准测试集。通过PROM和TSMD-Bench的实验表明：PROM能提供比现有指标更全面的评估；TSMD-Bench是比早期基准更具挑战性的测试平台；二者的结合有助于理解TSMD方法的相对性能。更广泛而言，所提出的方法为该领域实现大规模系统性性能比较提供了可能。