We investigate different natural language processing (NLP) approaches based on contextualised word representations for the problem of early prediction of lung cancer using free-text patient medical notes of Dutch primary care physicians. Because lung cancer has a low prevalence in primary care, we also address the problem of classification under highly imbalanced classes. Specifically, we use large Transformer-based pretrained language models (PLMs) and investigate: 1) how \textit{soft prompt-tuning} -- an NLP technique used to adapt PLMs using small amounts of training data -- compares to standard model fine-tuning; 2) whether simpler static word embedding models (WEMs) can be more robust compared to PLMs in highly imbalanced settings; and 3) how models fare when trained on notes from a small number of patients. We find that 1) soft-prompt tuning is an efficient alternative to standard model fine-tuning; 2) PLMs show better discrimination but worse calibration compared to simpler static word embedding models as the classification problem becomes more imbalanced; and 3) results when training models on small number of patients are mixed and show no clear differences between PLMs and WEMs. All our code is available open source in \url{https://bitbucket.org/aumc-kik/prompt_tuning_cancer_prediction/}.

翻译：我们研究了基于上下文化词表示的不同自然语言处理（NLP）方法，用于利用荷兰初级保健医生的自由文本患者医疗记录进行肺癌早期预测。由于肺癌在初级保健中发病率较低，我们还解决了高度不平衡分类问题。具体而言，我们使用基于Transformer的大型预训练语言模型（PLMs），并探讨：1）软提示调优（一种利用少量训练数据适配PLMs的NLP技术）与标准模型微调相比效果如何；2）在高度不平衡设置中，简单的静态词嵌入模型（WEMs）是否比PLMs更具鲁棒性；3）当使用少量患者记录训练模型时，各模型表现如何。我们发现：1）软提示调优是标准模型微调的高效替代方案；2）随着分类问题不平衡程度增加，PLMs相比简单的静态词嵌入模型表现出更好的判别能力但更差的校准能力；3）使用少量患者记录训练模型的结果参差不齐，PLMs与WEMs之间无明显差异。我们的所有代码均开源，可通过 \url{https://bitbucket.org/aumc-kik/prompt_tuning_cancer_prediction/} 获取。