Clinical prediction is an essential task in the healthcare industry. However, the recent success of transformers, on which large language models are built, has not been extended to this domain. In this research, we explore the use of transformers and language models in prognostic prediction for immunotherapy using real-world patients' clinical data and molecular profiles. This paper investigates the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and addresses the challenge of few-shot learning in predicting rare disease areas. The study benchmarks the efficacy of baselines and language models on prognostic prediction across multiple cancer types and investigates the impact of different pretrained language models under few-shot regimes. The results demonstrate significant improvements in accuracy and highlight the potential of NLP in clinical research to improve early detection and intervention for different diseases.

翻译：临床预测是医疗行业中的一项关键任务。然而，近期Transformer（大型语言模型的核心架构）的成功尚未扩展至该领域。本研究探索了基于真实患者临床数据与分子图谱，利用Transformer和语言模型进行免疫治疗预后预测的可能性。本文考察了与传统机器学习方法相比，Transformer在优化临床预测方面的潜力，并针对罕见疾病预测中的少样本学习挑战展开研究。通过跨多种癌症类型的预后预测基准测试，本研究评估了基线模型与语言模型的效能，并探究了不同预训练语言模型在少样本场景下的影响。结果表明，准确率显著提升，凸显了自然语言处理在临床研究中改进不同疾病早期检测与干预的潜力。