Quantifying uncertainty in clinical predictions is critical for high-stakes diagnosis tasks. Conformal prediction offers a principled approach by providing prediction sets with theoretical coverage guarantees. However, in practice, patient distribution shifts violate the i.i.d. assumptions underlying standard conformal methods, leading to poor coverage in healthcare settings. In this work, we evaluate several conformal prediction approaches on EEG seizure classification, a task with known distribution shift challenges and label uncertainty. We demonstrate that personalized calibration strategies can improve coverage by over 20 percentage points while maintaining comparable prediction set sizes. Our implementation is available via PyHealth, an open-source healthcare AI framework: https://github.com/sunlabuiuc/PyHealth.

翻译：在临床预测中量化不确定性对于高风险诊断任务至关重要。共形预测通过提供具有理论覆盖保证的预测集，提供了一种原则性方法。然而，在实际应用中，患者分布偏移违反了标准共形方法所依据的独立同分布假设，导致在医疗环境中的覆盖效果不佳。在本研究中，我们评估了几种共形预测方法在脑电图癫痫分类任务中的表现，该任务存在已知的分布偏移挑战和标签不确定性。我们证明，个性化校准策略可以将覆盖效果提高超过20个百分点，同时保持相当的预测集大小。我们的实现可通过PyHealth（一个开源医疗AI框架）获取：https://github.com/sunlabuiuc/PyHealth。