Fine-tuning large language models for domain-specific tasks such as medical text summarization demands substantial computational resources. Parameter-efficient fine-tuning (PEFT) methods offer promising alternatives by updating only a small fraction of parameters. This paper compares three adaptation approaches-Low-Rank Adaptation (LoRA), Prompt Tuning, and Full Fine-Tuning-across the Flan-T5 model family on the PubMed medical summarization dataset. Through experiments with multiple random seeds, we demonstrate that LoRA consistently outperforms full fine-tuning, achieving 43.52 +/- 0.18 ROUGE-1 on Flan-T5-Large with only 0.6% trainable parameters compared to 40.67 +/- 0.21 for full fine-tuning. Sensitivity analyses examine the impact of LoRA rank and prompt token count. Our findings suggest the low-rank constraint provides beneficial regularization, challenging assumptions about the necessity of full parameter updates. Code is available at https://github.com/eracoding/llm-medical-summarization

翻译：针对医学文本摘要等特定领域任务微调大型语言模型需要大量计算资源。参数高效微调（PEFT）方法通过仅更新少量参数提供了有前景的替代方案。本文在PubMed医学摘要数据集上，对Flan-T5模型家族的三种适配方法——低秩适配（LoRA）、提示微调与全参数微调进行了比较。通过多重随机种子的实验，我们证明LoRA始终优于全参数微调：在仅使用0.6%可训练参数的情况下，Flan-T5-Large模型上LoRA达到43.52 ± 0.18的ROUGE-1分数，而全参数微调仅获得40.67 ± 0.21。敏感性分析考察了LoRA秩与提示令牌数量对性能的影响。我们的研究结果表明，低秩约束提供了有益的正则化效果，这对全参数更新的必要性假设提出了挑战。代码已开源至https://github.com/eracoding/llm-medical-summarization