Time Series Foundation Models (TSFMs) are transforming the field of forecasting. However, evaluating them on historical data is increasingly difficult due to the risks of train-test sample overlaps and temporal overlaps between correlated train and test time series. To address this, we introduce TS-Arena, a live forecasting platform that shifts evaluation from the known past to the unknown future. Building on the concept of continuous benchmarking, TS-Arena evaluates models on future data. Crucially, we introduce a strict forecasting pre-registration protocol: models must submit predictions before the ground-truth data physically exists. This makes test-set contamination impossible by design. The platform relies on a modular microservice architecture that harmonizes and structures data from different sources and orchestrates containerized model submissions. By enforcing a strict pre-registration protocol on live data streams, TS-Arena prevents information leakage offers a faster alternative to traditional static, infrequently repeated competitions (e.g. the M-Competitions). First empirical results derived from operating TS-Arena over one year of energy time series demonstrate that established TSFMs accumulate robust longitudinal scores over time, while the continuous nature of the benchmark simultaneously allows newcomers to demonstrate immediate competitiveness. TS-Arena provides the necessary infrastructure to assess the true generalization capabilities of modern forecasting models. The platform and corresponding code are available at https://ts-arena.live/.
翻译:时间序列基础模型(TSFMs)正在变革预测领域。然而,由于训练-测试样本重叠以及相关训练与测试时间序列间存在时间重叠的风险,基于历史数据评估这些模型日益困难。为解决此问题,我们推出了TS-Arena,一个将评估从已知过去转向未知未来的实时预测平台。基于持续基准测试的理念,TS-Arena在未来的数据上评估模型。关键在于,我们引入了严格的预测预注册协议:模型必须在真实数据实际存在之前提交预测结果。这从设计上杜绝了测试集污染的可能性。该平台采用模块化微服务架构,协调并结构化来自不同来源的数据,并编排容器化的模型提交。通过对实时数据流执行严格的预注册协议,TS-Arena防止了信息泄露,并为传统静态、低频重复的竞赛(如M-Competitions)提供了更快速的替代方案。基于TS-Arena运行一年能源时间序列的首批实证结果表明,成熟的TSFMs随时间积累出稳健的纵向评分,而基准测试的持续性同时允许新进模型展示即时竞争力。TS-Arena为评估现代预测模型的真实泛化能力提供了必要的基础设施。平台及相关代码可在https://ts-arena.live/获取。