While multimodal foundation models can now natively work with data beyond text, they remain underutilized in analyzing the considerable amounts of multi-dimensional time-series data in fields like healthcare, finance, and social sciences, representing a missed opportunity for richer, data-driven insights. This paper proposes a simple but effective method that leverages the existing vision encoders of these models to "see" time-series data via plots, avoiding the need for additional, potentially costly, model training. Our empirical evaluations show that this approach outperforms providing the raw time-series data as text, with the additional benefit that visual time-series representations demonstrate up to a 90% reduction in model API costs. We validate our hypothesis through synthetic data tasks of increasing complexity, progressing from simple functional form identification on clean data, to extracting trends from noisy scatter plots. To demonstrate generalizability from synthetic tasks with clear reasoning steps to more complex, real-world scenarios, we apply our approach to consumer health tasks - specifically fall detection, activity recognition, and readiness assessment - which involve heterogeneous, noisy data and multi-step reasoning. The overall success in plot performance over text performance (up to an 120% performance increase on zero-shot synthetic tasks, and up to 150% performance increase on real-world tasks), across both GPT and Gemini model families, highlights our approach's potential for making the best use of the native capabilities of foundation models.

翻译：尽管多模态基础模型现已能够原生处理文本以外的数据，但在分析医疗、金融和社会科学等领域中大量存在的多维时间序列数据时，这些模型仍未得到充分利用，这错失了获得更丰富数据驱动见解的机会。本文提出一种简单而有效的方法，利用这些模型现有的视觉编码器通过图表来"观察"时间序列数据，从而避免额外且可能成本高昂的模型训练。我们的实证评估表明，该方法优于将原始时间序列数据作为文本输入的方式，并具有额外优势：视觉化时间序列表示可使模型API成本降低高达90%。我们通过复杂度递增的合成数据任务验证了该假设，从清洁数据中的简单函数形式识别，逐步推进到从噪声散点图中提取趋势。为证明从具有清晰推理步骤的合成任务到更复杂现实场景的泛化能力，我们将该方法应用于消费者健康任务——具体包括跌倒检测、活动识别和就绪状态评估——这些任务涉及异构噪声数据和多步推理。在GPT和Gemini模型系列中，图表性能相较于文本性能的整体优势（零样本合成任务性能提升最高达120%，现实任务性能提升最高达150%）凸显了我们这种方法在充分利用基础模型原生能力方面的潜力。