In this work, we leverage pre-trained Large Language Models (LLMs) to enhance time-series forecasting. Mirroring the growing interest in unifying models for Natural Language Processing and Computer Vision, we envision creating an analogous model for long-term time-series forecasting. Due to limited large-scale time-series data for building robust foundation models, our approach LLM4TS focuses on leveraging the strengths of pre-trained LLMs. By combining time-series patching with temporal encoding, we have enhanced the capability of LLMs to handle time-series data effectively. Inspired by the supervised fine-tuning in chatbot domains, we prioritize a two-stage fine-tuning process: first conducting supervised fine-tuning to orient the LLM towards time-series data, followed by task-specific downstream fine-tuning. Furthermore, to unlock the flexibility of pre-trained LLMs without extensive parameter adjustments, we adopt several Parameter-Efficient Fine-Tuning (PEFT) techniques. Drawing on these innovations, LLM4TS has yielded state-of-the-art results in long-term forecasting. Our model has also shown exceptional capabilities as both a robust representation learner and an effective few-shot learner, thanks to the knowledge transferred from the pre-trained LLM.

翻译：本研究利用预训练大语言模型提升时间序列预测能力。顺应自然语言处理与计算机视觉领域模型统一化的趋势，我们致力于构建面向长期时间序列预测的类比模型。针对大规模时间序列数据匮乏难以构建稳健基础模型的困境，本文提出的LLM4TS方法聚焦于发挥预训练大语言模型的优势。通过将时间序列分块技术与时序编码相结合，我们显著增强了预训练大语言模型处理时间序列数据的能力。受聊天机器人领域监督微调方法的启发，我们优先采用两阶段微调流程：首先通过监督微调引导大语言模型适应时间序列数据，随后进行任务特定下游微调。为在不进行大量参数调整的前提下释放预训练大语言模型的灵活性，我们引入了多种参数高效微调技术。基于上述创新，LLM4TS在长期预测任务中取得了当前最优性能。得益于预训练大语言模型的知识迁移，该模型既展现出作为鲁棒表示学习器的卓越能力，又在小样本学习场景中表现出色。