Clinical texts, such as admission notes, discharge summaries, and progress notes, contain rich and valuable information that can be used for various clinical outcome prediction tasks. However, applying large language models, such as BERT-based models, to clinical texts poses two major challenges: the limitation of input length and the diversity of data sources. This paper proposes a novel method to preserve the knowledge of long clinical texts using aggregated ensembles of large language models. Unlike previous studies which use model ensembling or text aggregation methods separately, we combine ensemble learning with text aggregation and train multiple large language models on two clinical outcome tasks: mortality prediction and length of stay prediction. We show that our method can achieve better results than baselines, ensembling, and aggregation individually, and can improve the performance of large language models while handling long inputs and diverse datasets. We conduct extensive experiments on the admission notes from the MIMIC-III clinical database by combining multiple unstructured and high-dimensional datasets, demonstrating our method's effectiveness and superiority over existing approaches. We also provide a comprehensive analysis and discussion of our results, highlighting our method's applications and limitations for future research in the domain of clinical healthcare. The results and analysis of this study is supportive of our method assisting in clinical healthcare systems by enabling clinical decision-making with robust performance overcoming the challenges of long text inputs and varied datasets.

翻译：临床文本（如入院记录、出院小结和病程记录）蕴含丰富且有价值的信息，可用于多种临床结局预测任务。然而，将BERT等基于大型语言模型的模型应用于临床文本面临两大挑战：输入长度限制与数据源多样性。本文提出一种创新方法，通过聚合集成大型语言模型来保留长临床文本的知识。与以往分别采用模型集成或文本聚合方法的研究不同，我们将集成学习与文本聚合相结合，针对两项临床结局任务（死亡率预测和住院时长预测）训练多个大型语言模型。实验表明，我们的方法在基线方法、单独集成方法和单独聚合方法上均能取得更优结果，同时能够在处理长文本输入和多样化数据集时提升大型语言模型的性能。我们通过整合MIMIC-III临床数据库中多个非结构化高维数据集的入院记录进行广泛实验，证明了该方法相对于现有方法的有效性和优越性。此外，我们对实验结果进行了全面分析与讨论，阐明了该方法在临床医疗领域未来研究中的应用价值与局限性。本研究的结果与分析支持该方法通过克服长文本输入和多样化数据集的挑战，以稳健性能辅助临床决策，助力临床医疗系统。