Time series data is ubiquitous in real-world scenarios and crucial for critical applications ranging from energy management to traffic control. Consequently, the ability to reason over time series is a fundamental skill for generalist models to solve practical problems. However, this dimension is notably absent from existing benchmarks of generalist models. To bridge this gap, we introduce TSRBench, a comprehensive multi-modal benchmark designed to stress-test the full spectrum of time series reasoning capabilities. TSRBench features: i) a diverse set of 4125 problems from 14 domains, and is categorized into 4 major dimensions: Perception, Reasoning, Prediction, and Decision-Making. ii) 15 tasks from the 4 dimensions evaluating essential reasoning capabilities (e.g., numerical reasoning). Through extensive experiments, we evaluated over 30 leading proprietary and open-source LLMs, VLMs, and TSLLMs within TSRBench. Our findings reveal that: i) scaling laws hold for perception and reasoning but break down for prediction; ii) strong reasoning does not guarantee accurate context-aware forecasting, indicating a decoupling between semantic understanding and numerical prediction; and iii) despite the complementary nature of textual and visual represenations of time series as inputs, current multimodal models fail to effectively fuse them for reciprocal performance gains. TSRBench provides a standardized evaluation platform that not only highlights existing challenges but also offers valuable insights to advance generalist models. Our code and dataset are available at https://tsrbench.github.io/.

翻译：时间序列数据在现实场景中无处不在，对于从能源管理到交通控制等关键应用至关重要。因此，对时间序列进行推理的能力是通用模型解决实际问题的基本技能。然而，现有通用模型基准显著缺乏对这一维度的评估。为弥补这一空白，我们引入了TSRBench，一个全面的多模态基准，旨在对时间序列推理能力的全谱系进行压力测试。TSRBench具有以下特点：i) 包含来自14个领域的4125个多样化问题，并归类为4个主要维度：感知、推理、预测和决策。ii) 涵盖这4个维度的15项任务，用于评估基本推理能力（例如数值推理）。通过大量实验，我们在TSRBench中评估了超过30个领先的专有及开源LLM、VLM和TSLLM。我们的研究发现：i) 缩放定律在感知和推理维度成立，但在预测维度失效；ii) 强大的推理能力并不能保证准确的上下文感知预测，表明语义理解与数值预测之间存在解耦；iii) 尽管时间序列的文本和视觉表示作为输入具有互补性，当前的多模态模型未能有效融合它们以获得相互的性能提升。TSRBench提供了一个标准化的评估平台，不仅突显了现有挑战，也为推进通用模型的发展提供了有价值的见解。我们的代码和数据集可在 https://tsrbench.github.io/ 获取。