Lung nodule malignancy prediction has been enhanced by advanced deep-learning techniques and effective tricks. Nevertheless, current methods are mainly trained with cross-entropy loss using one-hot categorical labels, which results in difficulty in distinguishing those nodules with closer progression labels. Interestingly, we observe that clinical text information annotated by radiologists provides us with discriminative knowledge to identify challenging samples. Drawing on the capability of the contrastive language-image pre-training (CLIP) model to learn generalized visual representations from text annotations, in this paper, we propose CLIP-Lung, a textual knowledge-guided framework for lung nodule malignancy prediction. First, CLIP-Lung introduces both class and attribute annotations into the training of the lung nodule classifier without any additional overheads in inference. Second, we designed a channel-wise conditional prompt (CCP) module to establish consistent relationships between learnable context prompts and specific feature maps. Third, we align image features with both class and attribute features via contrastive learning, rectifying false positives and false negatives in latent space. The experimental results on the benchmark LIDC-IDRI dataset have demonstrated the superiority of CLIP-Lung, both in classification performance and interpretability of attention maps.

翻译：肺结节恶性预测已通过先进的深度学习技术和有效技巧得到提升。然而，当前方法主要使用独热编码的类别标签通过交叉熵损失进行训练，导致难以区分进展标签相近的结节。有趣的是，我们观察到放射科医生标注的临床文本信息能提供识别困难样本的判别性知识。受对比语言-图像预训练（CLIP）模型从文本标注中学习通用视觉表示能力的启发，本文提出CLIP-Lung——一种文本知识引导的肺结节恶性预测框架。首先，CLIP-Lung将类别和属性标注引入肺结节分类器的训练，而无需在推理时增加额外开销。其次，我们设计了一个通道条件提示（CCP）模块，以建立可学习上下文提示与特定特征图之间的一致关系。第三，通过对比学习对齐图像特征与类别及属性特征，修正潜在空间中的假阳性和假阴性。在基准LIDC-IDRI数据集上的实验结果表明，CLIP-Lung在分类性能和注意力图可解释性上均具优越性。