Understanding time series is crucial for its application in real-world scenarios. Recently, large language models (LLMs) have been increasingly applied to time series tasks, leveraging their strong language capabilities to enhance various applications. However, research on multimodal LLMs (MLLMs) for time series understanding and reasoning remains limited, primarily due to the scarcity of high-quality datasets that align time series with textual information. This paper introduces ChatTS, a novel MLLM designed for time series analysis. ChatTS treats time series as a modality, similar to how vision MLLMs process images, enabling it to perform both understanding and reasoning with time series. To address the scarcity of training data, we propose an attribute-based method for generating synthetic time series with detailed attribute descriptions. We further introduce Time Series Evol-Instruct, a novel approach that generates diverse time series Q&As, enhancing the model's reasoning capabilities. To the best of our knowledge, ChatTS is the first MLLM that takes multivariate time series as input, which is fine-tuned exclusively on synthetic datasets. We evaluate its performance using benchmark datasets with real-world data, including six alignment tasks and four reasoning tasks. Our results show that ChatTS significantly outperforms existing vision-based MLLMs (e.g., GPT-4o) and text/agent-based LLMs, achieving a 46.0% improvement in alignment tasks and a 25.8% improvement in reasoning tasks.

翻译：理解时间序列对其在现实场景中的应用至关重要。近年来，大语言模型（LLMs）越来越多地应用于时间序列任务，利用其强大的语言能力来增强各种应用。然而，用于时间序列理解和推理的多模态大语言模型（MLLMs）研究仍然有限，这主要源于高质量的时间序列与文本信息对齐数据集的稀缺。本文介绍了ChatTS，一种专为时间序列分析设计的新型MLLM。ChatTS将时间序列视为一种模态，类似于视觉MLLMs处理图像的方式，使其能够对时间序列执行理解和推理任务。为了解决训练数据稀缺的问题，我们提出了一种基于属性的方法，用于生成具有详细属性描述的合成时间序列。我们进一步引入了Time Series Evol-Instruct，一种生成多样化时间序列问答的新方法，以增强模型的推理能力。据我们所知，ChatTS是首个以多元时间序列作为输入、并完全在合成数据集上进行微调的MLLM。我们使用包含真实数据的基准数据集评估其性能，包括六项对齐任务和四项推理任务。我们的结果表明，ChatTS显著优于现有的基于视觉的MLLMs（例如GPT-4o）以及基于文本/代理的LLMs，在对齐任务中实现了46.0%的提升，在推理任务中实现了25.8%的提升。