We systematically investigate lightweight strategies to adapt large language models (LLMs) for the task of radiology report summarization (RRS). Specifically, we focus on domain adaptation via pretraining (on natural language, biomedical text, and clinical text) and via prompting (zero-shot, in-context learning) or parameter-efficient fine-tuning (prefix tuning, LoRA). Our results on the MIMIC-III dataset consistently demonstrate best performance by maximally adapting to the task via pretraining on clinical text and parameter-efficient fine-tuning on RRS examples. Importantly, this method fine-tunes a mere 0.32% of parameters throughout the model, in contrast to end-to-end fine-tuning (100% of parameters). Additionally, we study the effect of in-context examples and out-of-distribution (OOD) training before concluding with a radiologist reader study and qualitative analysis. Our findings highlight the importance of domain adaptation in RRS and provide valuable insights toward developing effective natural language processing solutions for clinical tasks.

翻译：我们系统研究了将大型语言模型（LLMs）适配于放射学报告摘要（RRS）任务的轻量级策略。具体而言，我们重点探索了通过预训练（基于自然语言、生物医学文本与临床文本）、提示学习（零样本、上下文学习）或参数高效微调（前缀微调、LoRA）实现的领域适应方法。在MIMIC-III数据集上的结果一致表明，通过临床文本预训练与RRS示例参数高效微调的最大化任务适配策略表现最优。值得注意的是，该方法仅微调了整个模型中0.32%的参数，而端到端微调需调整全部参数。此外，我们研究了上下文示例及分布外（OOD）训练的影响，并通过放射科医师读者研究与定性分析验证结论。本研究成果揭示了领域适应在RRS中的关键作用，为开发临床任务的有效自然语言处理解决方案提供了重要启示。