In this work, we leverage pre-trained Large Language Models (LLMs) to enhance time-series forecasting. Mirroring the growing interest in unifying models for Natural Language Processing and Computer Vision, we envision creating an analogous model for long-term time-series forecasting. Due to limited large-scale time-series data for building robust foundation models, our approach LLM4TS focuses on leveraging the strengths of pre-trained LLMs. By combining time-series patching with temporal encoding, we have enhanced the capability of LLMs to handle time-series data effectively. Inspired by the supervised fine-tuning in chatbot domains, we prioritize a two-stage fine-tuning process: first conducting supervised fine-tuning to orient the LLM towards time-series data, followed by task-specific downstream fine-tuning. Furthermore, to unlock the flexibility of pre-trained LLMs without extensive parameter adjustments, we adopt several Parameter-Efficient Fine-Tuning (PEFT) techniques. Drawing on these innovations, LLM4TS has yielded state-of-the-art results in long-term forecasting. Our model has also shown exceptional capabilities as both a robust representation learner and an effective few-shot learner, thanks to the knowledge transferred from the pre-trained LLM.

翻译：本文利用预训练大语言模型（LLMs）来增强时间序列预测能力。借鉴自然语言处理与计算机视觉领域统一模型的持续发展趋势，我们设想构建面向长期时间序列预测的类比模型。由于缺乏大规模时间序列数据来构建鲁棒的基础模型，我们提出的LLM4TS方法聚焦于发挥预训练LLMs的优势。通过将时间序列分块技术与时间编码相结合，我们有效提升了LLMs处理时间序列数据的能力。受聊天机器人领域有监督微调的启发，我们优先采用两阶段微调流程：首先进行有监督微调使LLM适应时间序列数据，随后进行任务特定的下游微调。此外，为释放预训练LLMs的灵活性而无需大量参数调整，我们采用了多项参数高效微调（PEFT）技术。基于这些创新，LLM4TS在长期预测任务中取得了最先进的成果。得益于预训练LLMs的知识迁移能力，我们的模型还展现出作为鲁棒表征学习器和高效少样本学习器的卓越性能。