Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds.
翻译:将深度学习模型部署在经认证的现实系统中,需要具备精准反映其不确定性的置信度评估能力。本文通过共形预测框架构建可靠可信的铁路信号检测预测器。该方法基于包含列车驾驶员视角图像的新型数据集与现代目标检测器。我们测试了多种共形方法,并引入基于共形风险控制的新技术。研究结果证实了共形预测框架在评估模型性能方面的潜力,为获得具有形式化保证的不确定性边界提供了实践指导。