Uncertainty estimation for unlabeled data is crucial to active learning. With a deep neural network employed as the backbone model, the data selection process is highly challenging due to the potential over-confidence of the model inference. Existing methods resort to special learning fashions (e.g. adversarial) or auxiliary models to address this challenge. This tends to result in complex and inefficient pipelines, which would render the methods impractical. In this work, we propose a novel algorithm that leverages noise stability to estimate data uncertainty. The key idea is to measure the output derivation from the original observation when the model parameters are randomly perturbed by noise. We provide theoretical analyses by leveraging the small Gaussian noise theory and demonstrate that our method favors a subset with large and diverse gradients. Our method is generally applicable in various tasks, including computer vision, natural language processing, and structural data analysis. It achieves competitive performance compared against state-of-the-art active learning baselines.

翻译：未标记数据的不确定性估计对主动学习至关重要。当深度神经网络作为骨干模型时，由于模型推理可能存在的过度置信问题，数据选择过程极具挑战性。现有方法通过采用特殊学习范式（如对抗性学习）或辅助模型来应对这一挑战。这往往导致流程复杂且低效，使方法难以实际应用。本文提出一种利用噪声稳定性估计数据不确定性的新颖算法。核心思想是：当模型参数被随机噪声扰动时，测量原始观测值的输出偏差。我们通过引入小高斯噪声理论进行理论分析，证明该方法倾向于选择梯度幅度大且多样性高的子集。本方法可广泛适用于计算机视觉、自然语言处理及结构化数据分析等各类任务，与当前最优的主动学习基线方法相比，取得了具有竞争力的性能。