Low-Rank Adaptation (LoRA) is a widely used technique for fine-tuning large pre-trained or foundational models across different modalities and tasks. However, its application to time series data, particularly within foundational models, remains underexplored. This paper examines the impact of LoRA on contemporary time series foundational models: Lag-Llama, MOIRAI, and Chronos. We demonstrate LoRA's fine-tuning potential for forecasting the vital signs of sepsis patients in intensive care units (ICUs), emphasizing the models' adaptability to previously unseen, out-of-domain modalities. Integrating LoRA aims to enhance forecasting performance while reducing inefficiencies associated with fine-tuning large models on limited domain-specific data. Our experiments show that LoRA fine-tuning of time series foundational models significantly improves forecasting, achieving results comparable to state-of-the-art models trained from scratch on similar modalities. We conduct comprehensive ablation studies to demonstrate the trade-offs between the number of tunable parameters and forecasting performance and assess the impact of varying LoRA matrix ranks on model performance.

翻译：低秩适配（LoRA）是一种广泛用于跨模态和任务场景微调大型预训练或基础模型的技术。然而，其在时间序列数据（尤其是基础模型中的应用）仍处于未被充分探索的阶段。本文研究了LoRA对当代时间序列基础模型（Lag-Llama、MOIRAI和Chronos）的影响。我们展示了LoRA在重症监护病房（ICU）脓毒症患者生命体征预测中的微调潜力，强调了模型对未曾见过的域外模态的适应性。集成LoRA旨在提升预测性能，同时缓解在有限领域特定数据上微调大型模型时产生的低效问题。实验表明，对时间序列基础模型进行LoRA微调能显著改进预测效果，其结果可媲美在相似模态上从头训练的顶尖模型。我们通过全面的消融研究揭示了可调参数数量与预测性能之间的权衡，并评估了不同LoRA矩阵秩对模型性能的影响。