In this study, we present aLLM4TS, an innovative framework that adapts Large Language Models (LLMs) for time-series representation learning. Central to our approach is that we reconceive time-series forecasting as a self-supervised, multi-patch prediction task, which, compared to traditional mask-and-reconstruction methods, captures temporal dynamics in patch representations more effectively. Our strategy encompasses two-stage training: (i). a causal continual pre-training phase on various time-series datasets, anchored on next patch prediction, effectively syncing LLM capabilities with the intricacies of time-series data; (ii). fine-tuning for multi-patch prediction in the targeted time-series context. A distinctive element of our framework is the patch-wise decoding layer, which departs from previous methods reliant on sequence-level decoding. Such a design directly transposes individual patches into temporal sequences, thereby significantly bolstering the model's proficiency in mastering temporal patch-based representations. aLLM4TS demonstrates superior performance in several downstream tasks, proving its effectiveness in deriving temporal representations with enhanced transferability and marking a pivotal advancement in the adaptation of LLMs for time-series analysis.

翻译：在本研究中，我们提出了aLLM4TS，一个创新的框架，用于适配大型语言模型（LLMs）进行时间序列表示学习。我们方法的核心是将时间序列预测重新构想为一种自监督的多补丁预测任务，与传统掩码-重构方法相比，该方法能更有效地捕捉补丁表示中的时间动态。我们的策略包括两个阶段的训练：(i) 一种基于下一补丁预测的因果连续预训练阶段，在多种时间序列数据集上进行，有效将LLM的能力与时间序列数据的复杂性同步；(ii) 针对特定时间序列上下文的微调，用于多补丁预测。我们框架的一个独特元素是逐补丁解码层，这与以往依赖序列级解码的方法不同。这种设计直接将单个补丁转换为时间序列，从而显著增强了模型在掌握基于补丁的时间表示方面的能力。aLLM4TS在多个下游任务中表现出优越性能，证明其在生成具有增强可迁移性的时间表示方面的有效性，标志着LLMs适应时间序列分析的关键进展。