Clinical prediction is an essential task in the healthcare industry. However, the recent success of transformers, on which large language models are built, has not been extended to this domain. In this research, we explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. This paper investigates the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and addresses the challenge of few-shot learning in predicting rare disease areas. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes. The results demonstrate significant improvements in accuracy and highlight the potential of NLP in clinical research to improve early detection and intervention for different diseases.

翻译：临床预测是医疗行业中的一项关键任务。然而，近年来基于Transformer架构的大型语言模型取得的成功尚未扩展至该领域。本研究探索了利用Transformer和语言模型，结合真实世界患者的临床数据与分子谱，在免疫治疗中进行预后预测。本文比较了Transformer与传统机器学习方法在临床预测中的潜力，并针对罕见疾病预测中的小样本学习挑战进行了研究。研究在多癌种预后预测上基准测试了基线方法与语言模型的效能，并分析了不同预训练语言模型在小样本场景下的影响。结果表明，准确率显著提升，凸显了自然语言处理在临床研究中改善不同疾病早期检测与干预的潜力。