Uveitis demands the precise diagnosis of anterior chamber inflammation (ACI) for optimal treatment. However, current diagnostic methods only rely on a limited single-modal disease perspective, which leads to poor performance. In this paper, we investigate a promising yet challenging way to fuse multimodal data for ACI diagnosis. Notably, existing fusion paradigms focus on empowering implicit modality interactions (i.e., self-attention and its variants), but neglect to inject explicit modality interactions, especially from clinical knowledge and imaging property. To this end, we propose a jointly Explicit and implicit Cross-Modal Interaction Network (EiCI-Net) for Anterior Chamber Inflammation Diagnosis that uses anterior segment optical coherence tomography (AS-OCT) images, slit-lamp images, and clinical data jointly. Specifically, we first develop CNN-Based Encoders and Tabular Processing Module (TPM) to extract efficient feature representations in different modalities. Then, we devise an Explicit Cross-Modal Interaction Module (ECIM) to generate attention maps as a kind of explicit clinical knowledge based on the tabular feature maps, then integrated them into the slit-lamp feature maps, allowing the CNN-Based Encoder to focus on more effective informativeness of the slit-lamp images. After that, the Implicit Cross-Modal Interaction Module (ICIM), a transformer-based network, further implicitly enhances modality interactions. Finally, we construct a considerable real-world dataset from our collaborative hospital and conduct sufficient experiments to demonstrate the superior performance of our proposed EiCI-Net compared with the state-of-the-art classification methods in various metrics.

翻译：葡萄膜炎需要精确诊断眼前房炎症以实现最佳治疗。然而，当前诊断方法仅依赖有限的单模态疾病视角，导致性能不佳。本文探索了一种有前景且具有挑战性的多模态数据融合方法用于眼前房炎症诊断。值得注意的是，现有融合范式侧重于增强隐式模态交互（如自注意力机制及其变体），但忽略了注入显式模态交互，特别是来自临床知识和成像特性的交互。为此，我们提出了一种联合显式和隐式跨模态交互网络（EiCI-Net），用于眼前房炎症诊断，该网络联合使用眼前段光学相干断层扫描（AS-OCT）图像、裂隙灯图像和临床数据。具体而言，我们首先开发基于CNN的编码器和表格处理模块（TPM）以提取不同模态的有效特征表示。然后，我们设计显式跨模态交互模块（ECIM），基于表格特征图生成注意力图作为一类显式临床知识，并将其集成到裂隙灯特征图中，使基于CNN的编码器能够聚焦于裂隙灯图像中更有效的信息。随后，隐式跨模态交互模块（ICIM）作为基于Transformer的网络，进一步隐式增强模态交互。最后，我们从合作医院构建了大规模真实世界数据集，并通过充分实验证明，所提出的EiCI-Net在各种指标上均优于最先进的分类方法。