Reliable uncertainty estimation is critical for medical image segmentation, where automated contours feed downstream quantification and clinical decision support. Many strong uncertainty methods require repeated inference, while efficient single-forward-pass alternatives often provide weaker failure ranking or rely on restrictive feature-space assumptions. We present $\textbf{SegWithU}$, a post-hoc framework that augments a frozen pretrained segmentation backbone with a lightweight uncertainty head. SegWithU taps intermediate backbone features and models uncertainty as perturbation energy in a compact probe space using rank-1 posterior probes. It produces two voxel-wise uncertainty maps: a calibration-oriented map for probability tempering and a ranking-oriented map for error detection and selective prediction. Across ACDC, BraTS2024, and LiTS, SegWithU is the strongest and most consistent single-forward-pass baseline, achieving AUROC/AURC of $0.9838/2.4885$, $0.9946/0.2660$, and $0.9925/0.8193$, respectively, while preserving segmentation quality. These results suggest that perturbation-based uncertainty modeling is an effective and practical route to reliability-aware medical segmentation. Source code is available at https://github.com/ProjectNeura/SegWithU.

翻译：可靠的 uncertainty 估计对于医学图像分割至关重要，因为自动化轮廓为下游量化分析和临床决策支持提供输入。许多强大的不确定性方法需要重复推理，而高效的单次前向传播替代方法往往提供较弱的失效排序能力，或依赖有局限性的特征空间假设。我们提出 $\textbf{SegWithU}$，这是一种事后框架，通过轻量级不确定性头增强冻结的预训练分割骨干网络。SegWithU 利用中间骨干特征，并使用秩一后验探测器将不确定性建模为紧凑探针空间中的扰动能量。它生成两种体素级不确定性图：面向校准的概率温度调整图和面向排序的错误检测与选择性预测图。在 ACDC、BraTS2024 和 LiTS 数据集上，SegWithU 是最强且最一致的单次前向传播基线方法，分别实现了 $0.9838/2.4885$、$0.9946/0.2660$ 和 $0.9925/0.8193$ 的 AUROC/AURC 指标，同时保持了分割质量。这些结果表明，基于扰动的不确定性建模是实现可靠感知型医学分割的一种有效且实用的途径。源代码见 https://github.com/ProjectNeura/SegWithU。