The application of deep learning models to large-scale data sets requires means for automatic quality assurance. We have previously developed a fully automatic algorithm for carotid artery wall segmentation in black-blood MRI that we aim to apply to large-scale data sets. This method identifies nested artery walls in 3D patches centered on the carotid artery. In this study, we investigate to what extent the uncertainty in the model predictions for the contour location can serve as a surrogate for error detection and, consequently, automatic quality assurance. We express the quality of automatic segmentations using the Dice similarity coefficient. The uncertainty in the model's prediction is estimated using either Monte Carlo dropout or test-time data augmentation. We found that (1) including uncertainty measurements did not degrade the quality of the segmentations, (2) uncertainty metrics provide a good proxy of the quality of our contours if the center found during the first step is enclosed in the lumen of the carotid artery and (3) they could be used to detect low-quality segmentations at the participant level. This automatic quality assurance tool might enable the application of our model in large-scale data sets.

翻译：深度学习模型在大规模数据集中的应用需要自动质量保证手段。我们曾开发一种全自动算法，用于黑血MRI中颈动脉壁分割，旨在将其应用于大规模数据集。该方法可在以颈动脉为中心的3D图像块中识别嵌套的动脉壁。本研究探讨了模型预测轮廓位置的不确定性在多大程度上可作为误差检测的替代指标，进而实现自动质量保证。我们采用Dice相似系数量化自动分割质量。模型预测的不确定性通过蒙特卡洛丢弃法或测试时数据增强进行估计。研究发现：（1）引入不确定性测量不会降低分割质量；（2）若第一步定位的中心位于颈动脉管腔内，不确定性指标可有效反映轮廓质量；（3）该指标可识别参与者层面的低质量分割。此自动质量保证工具有望支持模型在大规模数据集中的应用。