Adapting Large Language Models (LLMs) that are extensively trained on abundant text data, and customizing the input prompt to enable time series forecasting has received considerable attention. While recent work has shown great potential for adapting the learned prior of LLMs, the formulation of the prompt to finetune LLMs remains challenging as prompt should be aligned with time series data. Additionally, current approaches do not effectively leverage word token embeddings which embody the rich representation space learned by LLMs. This emphasizes the need for a robust approach to formulate the prompt which utilizes the word token embeddings while effectively representing the characteristics of the time series. To address these challenges, we propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via LLMs. First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics via end-to-end finetuning. Next, we draw inspiration from Nearest Neighbor Contrastive Learning to formulate the prompt while obtaining the top-$k$ nearest neighbor time series compatible text prototypes. We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost. Our comprehensive experiments demonstrate that NNCL-TLLM outperforms in few-shot forecasting while achieving competitive or superior performance over the state-of-the-art methods in long-term and short-term forecasting tasks.

翻译：利用大规模文本数据充分训练的大语言模型（LLMs），并通过定制输入提示使其适用于时间序列预测任务，已引起广泛关注。尽管近期研究在适配LLMs所学先验知识方面展现出巨大潜力，但用于微调LLMs的提示构建仍具挑战性，因为提示需要与时间序列数据对齐。此外，现有方法未能有效利用体现LLMs丰富表征空间的词元嵌入。这凸显了需要一种鲁棒的方法来构建提示，既能利用词元嵌入，又能有效表征时间序列特征。为应对这些挑战，我们提出NNCL-TLLM：基于最近邻对比学习的大语言模型时间序列预测方法。首先，我们通过端到端微调生成时间序列兼容的文本原型，使得每个文本原型既能表征其邻域内的词元嵌入，又能体现时间序列特征。接着，我们借鉴最近邻对比学习思想，在获取前$k$个最近邻时间序列兼容文本原型的同时构建提示。随后，我们仅微调LLMs的层归一化与位置嵌入层，保持其他层不变，从而减少可训练参数并降低计算成本。综合实验表明，NNCL-TLLM在少样本预测任务中表现优异，同时在长期与短期预测任务中达到或超越了当前最优方法的性能。