Crohn's Disease (CD) and Ulcerative Colitis (UC) are the two main Inflammatory Bowel Disease (IBD) types. We developed deep learning models to identify histological disease features for both CD and UC using only endoscopic labels. We explored fine-tuning and end-to-end training of two state-of-the-art self-supervised models for predicting three different endoscopic categories (i) CD vs UC (AUC=0.87), (ii) normal vs lesional (AUC=0.81), (iii) low vs high disease severity score (AUC=0.80). We produced visual attention maps to interpret what the models learned and validated them with the support of a pathologist, where we observed a strong association between the models' predictions and histopathological inflammatory features of the disease. Additionally, we identified several cases where the model incorrectly predicted normal samples as lesional but were correct on the microscopic level when reviewed by the pathologist. This tendency of histological presentation to be more severe than endoscopic presentation was previously published in the literature. In parallel, we utilised a model trained on the Colon Nuclei Identification and Counting (CoNIC) dataset to predict and explore 6 cell populations. We observed correlation between areas enriched with the predicted immune cells in biopsies and the pathologist's feedback on the attention maps. Finally, we identified several cell level features indicative of disease severity in CD and UC. These models can enhance our understanding about the pathology behind IBD and can shape our strategies for patient stratification in clinical trials.

翻译：克罗恩病（CD）和溃疡性结肠炎（UC）是两种主要的炎性肠病（IBD）类型。我们开发了仅使用内镜标签即可识别CD和UC组织学疾病特征的深度学习模型。我们探索了两种先进自监督模型的微调和端到端训练，用于预测三种不同内镜类别：（i）CD与UC（AUC=0.87），（ii）正常与病变（AUC=0.81），（iii）低疾病严重程度评分与高疾病严重程度评分（AUC=0.80）。我们生成了视觉注意力图以解释模型所学内容，并在病理学家的支持下进行了验证，观察到模型预测与疾病组织病理学炎症特征之间存在强烈关联。此外，我们发现了多个模型将正常样本错误预测为病变的案例，但经病理学家复核后，这些样本在微观层面实际上确为病变。这种组织学表现比内镜表现更严重的趋势已有文献报道。与此同时，我们利用基于结肠细胞核识别与计数（CoNIC）数据集训练的模型预测并探索了6种细胞群体。我们观察到活检组织中预测免疫细胞富集区域与病理学家对注意力图的反馈之间存在相关性。最后，我们识别出了多个指示CD和UC疾病严重程度的细胞水平特征。这些模型可增强我们对IBD病理机制的理解，并优化临床试验中的患者分层策略。