Bone health studies are crucial in medical practice for the early detection and treatment of Osteopenia and Osteoporosis. Clinicians usually make a diagnosis based on densitometry (DEXA scans) and patient history. The applications of AI in this field are ongoing research. Most successful methods rely on deep learning models that use vision alone (DEXA/X-ray imagery) and focus on prediction accuracy, while explainability is often disregarded and left to post hoc assessments of input contributions. We propose ProtoMedX, a multi-modal model that uses both DEXA scans of the lumbar spine and patient records. ProtoMedX's prototype-based architecture is explainable by design, which is crucial for medical applications, especially in the context of the upcoming EU AI Act, as it allows explicit analysis of model decisions, including incorrect ones. ProtoMedX demonstrates state-of-the-art performance in bone health classification while also providing explanations that can be visually understood by clinicians. Using a dataset of 4,160 real NHS patients, the proposed ProtoMedX achieves 87.58% accuracy in vision-only tasks and 89.8% in its multi-modal variant, both surpassing existing published methods.

翻译：骨骼健康研究对于骨质疏松症和骨质减少症的早期检测与治疗具有重要临床意义。临床诊断通常基于骨密度检测（DEXA扫描）与患者病史。人工智能在该领域的应用正处于持续探索阶段。当前主流方法多依赖纯视觉深度学习模型（DEXA/X射线影像），侧重于预测准确性，而可解释性常被忽视，仅通过事后归因分析实现。本文提出ProtoMedX多模态模型，同时整合腰椎DEXA扫描与患者临床记录。该模型基于原型构建的架构具备内在可解释性，这对于医疗应用至关重要——尤其在欧盟《人工智能法案》即将实施的背景下，其支持对模型决策（包括错误判断）进行显式分析。ProtoMedX在实现最先进骨骼健康分类性能的同时，能够为临床医生提供可视化解释。基于4,160例真实NHS患者数据集的实验表明，所提方法在纯视觉任务中达到87.58%准确率，多模态版本达到89.8%，均超越现有公开方法。