Robotic systems often use predictive uncertainty to decide whether to act autonomously or defer to a fallback policy. In threshold-gated autonomy, uncertainty matters mainly through its ability to rank likely errors. Standard metrics such as expected calibration error and AUROC do not directly test whether uncertainty changes act/defer decisions. We therefore evaluate uncertainty using Spearman rank correlation, paired bootstrap equivalence testing, and act/defer agreement. Across three temporal activity-recognition benchmarks, we find a dataset-dependent competence regime below which uncertainty provides a weak and unstable error ranking. Above this regime, softmax heuristics, MC Dropout, and ensembles produce similar gating behavior, while threshold choice has a much larger effect on execution outcomes. A multi-seed embodied simulation shows the same pattern for collision rate and cost once realized autonomy is matched. Under temporal covariate shift, ranking quality remains stable, but fine grained semantic OOD detection remains near chance. These results suggest that simple uncertainty proxies can suffice for selective gating once the base model is competent, but not for semantic novelty detection.
翻译:机器人系统常利用预测不确定性来决定是自主行动还是退而求其次采用备用策略。在阈值门控自主性中,不确定性的主要作用体现为评估潜在排序错误的可能性。预期校准误差与AUROC等标准指标无法直接检验不确定性是否改变了"执行/放弃"决策。因此,我们采用斯皮尔曼秩相关系数、配对自助法等价性检验以及执行/放弃一致性评估不确定性。在三个时序活动识别基准测试中,我们发现存在一个与数据集相关的能力区间:低于该区间时,不确定性提供的是微弱且不稳定的错误排序;高于该区间时,softmax启发式方法、MC Dropout与集成方法产生相似的(不执行/执行)门控行为,而阈值选择对执行结果的影响更为显著。在匹配实际自主性后,多种子实体仿真在碰撞率和成本指标上呈现相同规律。面对时间协变量偏移时,排序质量保持稳定,但细粒度语义分布外检测仍接近随机水平。研究结果表明:一旦基础模型具备足够能力,简单的不确定性代理即可满足选择性门控需求,但无法胜任语义新颖性检测任务。