In this work, we leverage pre-trained Large Language Models (LLMs) to enhance time-series forecasting. Mirroring the growing interest in unifying models for Natural Language Processing and Computer Vision, we envision creating an analogous model for long-term time-series forecasting. Due to limited large-scale time-series data for building robust foundation models, our approach LLM4TS focuses on leveraging the strengths of pre-trained LLMs. By combining time-series patching with temporal encoding, we have enhanced the capability of LLMs to handle time-series data effectively. Inspired by the supervised fine-tuning in chatbot domains, we prioritize a two-stage fine-tuning process: first conducting supervised fine-tuning to orient the LLM towards time-series data, followed by task-specific downstream fine-tuning. Furthermore, to unlock the flexibility of pre-trained LLMs without extensive parameter adjustments, we adopt several Parameter-Efficient Fine-Tuning (PEFT) techniques. Drawing on these innovations, LLM4TS has yielded state-of-the-art results in long-term forecasting. Our model has also shown exceptional capabilities as both a robust representation learner and an effective few-shot learner, thanks to the knowledge transferred from the pre-trained LLM.

翻译：本文利用预训练大语言模型（LLMs）提升时间序列预测性能。针对自然语言处理与计算机视觉领域统一模型的趋势，我们探索构建面向长期时间序列预测的类比模型。由于大规模时间序列数据有限，难以构建稳健的基础模型，本文提出的LLM4TS方法聚焦于发挥预训练LLMs的优势。通过将时间序列分段与时间编码相结合，我们有效增强了LLMs处理时间序列数据的能力。受聊天机器人领域监督微调范式的启发，本文优先采用两阶段微调流程：首先通过监督微调引导LLMs适应时间序列数据，随后执行特定任务的下游微调。此外，为在不进行大量参数调整的情况下释放预训练LLMs的灵活性，我们采用多项参数高效微调（PEFT）技术。基于这些创新，LLM4TS在长期预测任务中取得了最优结果。得益于预训练LLMs的知识迁移，该模型在鲁棒表征学习与少样本学习方面均展现出卓越能力。