We introduce SLIMP (Skin Lesion Image-Metadata Pre-training) for learning rich representations of skin lesions through a novel nested contrastive learning approach that captures complex relationships between images and metadata. Melanoma detection and skin lesion classification based solely on images, pose significant challenges due to large variations in imaging conditions (lighting, color, resolution, distance, etc.) and lack of clinical and phenotypical context. Clinicians typically follow a holistic approach for assessing the risk level of the patient and for deciding which lesions may be malignant and need to be excised, by considering the patient's medical history as well as the appearance of other lesions of the patient. Inspired by this, SLIMP combines the appearance and the metadata of individual skin lesions with patient-level metadata relating to their medical record and other clinically relevant information. By fully exploiting all available data modalities throughout the learning process, the proposed pre-training strategy improves performance compared to other pre-training strategies on downstream skin lesions classification tasks highlighting the learned representations quality.

翻译：我们提出SLIMP（皮肤病变图像-元数据预训练）方法，通过一种新颖的嵌套对比学习框架来学习皮肤病变的丰富表征，该框架能够捕捉图像与元数据之间的复杂关系。仅基于图像的黑色素瘤检测和皮肤病变分类由于成像条件（光照、色彩、分辨率、距离等）的巨大差异以及临床和表型背景信息的缺失而面临重大挑战。临床医生通常采用整体性方法评估患者风险水平并判断哪些病变可能恶性需要切除，这需要综合考虑患者的医疗史及其其他病变的外观特征。受此启发，SLIMP将单个皮肤病变的外观特征和元数据，与患者医疗记录及其他临床相关信息的患者级元数据相结合。通过在学习过程中充分利用所有可用数据模态，所提出的预训练策略在下游皮肤病变分类任务中相比其他预训练策略表现出性能提升，这凸显了所学表征的质量优势。