Knowing when a trained segmentation model is encountering data that is different to its training data is important. Understanding and mitigating the effects of this play an important part in their application from a performance and assurance perspective - this being a safety concern in applications such as autonomous vehicles (AVs). This work presents a segmentation network that can detect errors caused by challenging test domains without any additional annotation in a single forward pass. As annotation costs limit the diversity of labelled datasets, we use easy-to-obtain, uncurated and unlabelled data to learn to perform uncertainty estimation by selectively enforcing consistency over data augmentation. To this end, a novel segmentation benchmark based on the SAX Dataset is used, which includes labelled test data spanning three autonomous-driving domains, ranging in appearance from dense urban to off-road. The proposed method, named Gamma-SSL, consistently outperforms uncertainty estimation and Out-of-Distribution (OoD) techniques on this difficult benchmark - by up to 10.7% in area under the receiver operating characteristic (ROC) curve and 19.2% in area under the precision-recall (PR) curve in the most challenging of the three scenarios.
翻译:了解训练后的分割模型何时遇到与训练数据不同的数据至关重要。理解和缓解其影响对其在性能与可靠性保障方面的应用具有重要作用——这在自动驾驶汽车等涉及安全的应用中尤为关键。本文提出一种分割网络,可在单次前向传播中无需额外标注即可检测由具有挑战性的测试域引起的错误。鉴于标注成本限制了标注数据集的多样性,我们利用易于获取、未经整理的无标注数据,通过学习对数据增强选择性施加一致性来实现不确定性估计。为此,基于SAX数据集构建了新型分割基准,其标注测试数据涵盖从密集城区到越野场景的三个自动驾驶域。所提出的方法名为Gamma-SSL,在该困难基准上持续优于不确定性估计和分布外检测技术——在最具挑战性的三个场景中,受试者工作特征曲线下面积提升最高达10.7%,精确率-召回率曲线下面积提升最高达19.2%。