Postoperative risk predictions can inform effective perioperative care management and planning. We aimed to assess whether clinical large language models (LLMs) can predict postoperative risks using clinical texts with various training strategies. The main cohort involved 84,875 records from Barnes Jewish Hospital (BJH) system between 2018 and 2021. Methods were replicated on Beth Israel Deaconess's MIMIC dataset. Both studies had mean duration of follow-up based on the length of postoperative ICU stay less than 7 days. For the BJH dataset, outcomes included 30-day mortality, pulmonary embolism (PE) and pneumonia. Three domain adaptation and finetuning strategies were implemented for BioGPT, ClinicalBERT and BioClinicalBERT: self-supervised objectives; incorporating labels with semi-supervised fine-tuning; and foundational modelling through multi-task learning. Model performance was compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC) for classification tasks, and mean squared error (MSE) and R2 for regression tasks. Pre-trained LLMs outperformed traditional word embeddings, with absolute maximal gains of 38.3% for AUROC and 14% for AUPRC. Adapting models further improved performance: (1) self-supervised finetuning by 3.2% for AUROC and 1.5% for AUPRC; (2) semi-supervised finetuning by 1.8% for AUROC and 2% for AUPRC, compared to self-supervised finetuning; (3) foundational modelling by 3.6% for AUROC and 2.6% for AUPRC, compared to self-supervised finetuning. Pre-trained clinical LLMs offer opportunities for postoperative risk predictions in unforeseen data, with peaks in foundational models indicating the potential of task-agnostic learning towards the generalizability of LLMs in perioperative care.

翻译：术后风险预测可为有效的围术期医疗管理与规划提供依据。本研究旨在评估基于临床文本的临床大语言模型（LLMs）在采用不同训练策略时预测术后风险的能力。主要队列纳入2018至2021年间巴恩斯-犹太医院（BJH）系统的84,875份病历记录，并在贝斯以色列女执事医疗中心的MIMIC数据集上复现研究方法。两项研究的平均随访时长均基于术后ICU住院时间不超过7天。针对BJH数据集，结局指标包括30天死亡率、肺栓塞（PE）及肺炎。我们为BioGPT、ClinicalBERT及BioClinicalBERT模型实施了三种领域自适应与微调策略：自监督目标学习、半监督标签微调及多任务学习基础建模。模型性能通过分类任务的受试者工作特征曲线下面积（AUROC）和精确率-召回率曲线下面积（AUPRC），以及回归任务的均方误差（MSE）和R²进行比较。预训练LLMs较传统词嵌入方法表现更优，AUROC与AUPRC分别实现38.3%和14%的最大绝对提升。模型自适应进一步改善性能：（1）自监督微调使AUROC提升3.2%、AUPRC提升1.5%；（2）相较自监督微调，半监督微调使AUROC提升1.8%、AUPRC提升2%；（3）相较自监督微调，基础建模使AUROC提升3.6%、AUPRC提升2.6%。预训练临床LLMs为未知数据的术后风险预测提供了新机遇，其中基础模型的峰值表现表明，任务无关学习有望推动LLMs在围术期医疗中的泛化能力。