Deep neural networks have great representation power, but typically require large numbers of training examples. This motivates deep active learning methods that can significantly reduce the amount of labeled training data. Empirical successes of deep active learning have been recently reported in the literature, however, rigorous label complexity guarantees of deep active learning have remained elusive. This constitutes a significant gap between theory and practice. This paper tackles this gap by providing the first near-optimal label complexity guarantees for deep active learning. The key insight is to study deep active learning from the nonparametric classification perspective. Under standard low noise conditions, we show that active learning with neural networks can provably achieve the minimax label complexity, up to disagreement coefficient and other logarithmic terms. When equipped with an abstention option, we further develop an efficient deep active learning algorithm that achieves $\mathsf{polylog}(\frac{1}ε)$ label complexity, without any low noise assumptions. We also provide extensions of our results beyond the commonly studied Sobolev/Hölder spaces and develop label complexity guarantees for learning in Radon $\mathsf{BV}^2$ spaces, which have recently been proposed as natural function spaces associated with neural networks.

翻译：深度神经网络具有强大的表示能力，但通常需要大量训练样本。这推动了深度主动学习方法的发展，其能显著减少标注训练数据的数量。近期文献中已有深度主动学习取得实证成功的报道，然而其严格的标注复杂度理论保证仍难以建立。这构成了理论与应用之间的显著差距。本文通过首次为深度主动学习提供近乎最优的标注复杂度保证来弥合这一差距。关键洞见在于从非参数分类的视角研究深度主动学习。在标准的低噪声条件下，我们证明基于神经网络的主动学习能够以可证明的方式达到极小极大标注复杂度，仅相差分歧系数及其他对数项。当配备弃权选项时，我们进一步提出一种高效的深度主动学习算法，该算法在无需任何低噪声假设的条件下即可实现$\mathsf{polylog}(\frac{1}ε)$的标注复杂度。我们还将研究结果扩展到常用的Sobolev/Hölder空间之外，并建立了在Radon $\mathsf{BV}^2$空间中学习的标注复杂度保证——该空间近期被提出作为与神经网络相关联的自然函数空间。