Finetuning Large Language Models helps improve the results for domain-specific use cases. End-to-end finetuning of large language models is time and resource intensive and has high storage requirements to store the finetuned version of the large language model. Parameter Efficient Fine Tuning (PEFT) methods address the time and resource challenges by keeping the large language model as a fixed base and add additional layers, which the PEFT methods finetune. This paper demonstrates the evaluation results for one such PEFT method Low Rank Adaptation (LoRA), for Clinical Dialogue Summarization. The evaluation results show that LoRA works at par with end-to-end finetuning for a large language model. The paper presents the evaluations done for solving both the Subtask A and B from ImageCLEFmedical {https://www.imageclef.org/2023/medical}
翻译:微调大型语言模型有助于提升特定领域用例的结果。大型语言模型的端到端微调既耗时又消耗资源,且需要大量存储空间来保存微调后的模型版本。参数高效微调方法通过保持大型语言模型作为固定基础并添加额外层来解决时间和资源挑战,而PEFT方法则对这些附加层进行微调。本文展示了一种PEFT方法——低秩适应在临床对话摘要中的评估结果。评估结果表明,LoRA在大型语言模型上的表现与端到端微调相当。本文还呈现了为解决ImageCLEFmedical(https://www.imageclef.org/2023/medical)中子任务A和B所进行的评估工作。