Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks crucial context necessary for accurate predictions. Human forecasters frequently rely on additional information, such as background knowledge or constraints, which can be efficiently communicated through natural language. However, the ability of existing forecasting models to effectively integrate this textual information remains an open question. To address this, we introduce "Context is Key" (CiK), a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context, requiring models to integrate both modalities. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters, and propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings. By presenting this benchmark, we aim to advance multimodal forecasting, promoting models that are both accurate and accessible to decision-makers with varied technical expertise. The benchmark can be visualized at https://servicenow.github.io/context-is-key-forecasting/v0/ .

翻译：预测是各领域决策中的关键任务。数值数据虽提供了基础，但往往缺乏准确预测所需的关键背景信息。人类预测者常依赖额外信息（如背景知识或约束条件），这些信息可通过自然语言高效传递。然而，现有预测模型能否有效整合此类文本信息仍是开放性问题。为此，我们提出"上下文是关键"（CiK）时序预测基准，该基准将数值数据与多种精心构建的文本语境配对，要求模型实现多模态整合。我们评估了包括统计模型、时序基础模型和基于LLM的预测器在内的多种方法，并提出一种简单而有效的LLM提示方法，该方法在我们的基准测试中优于所有其他测试方法。实验结果表明：整合语境信息至关重要；基于LLM的预测模型展现出令人惊讶的性能；同时也暴露出其若干关键缺陷。通过发布此基准，我们旨在推动多模态预测研究的发展，促进构建既准确又便于不同技术背景决策者使用的预测模型。基准可视化页面详见 https://servicenow.github.io/context-is-key-forecasting/v0/ 。