Previous work optimizes traditional active learning (AL) processes with incremental neural network architecture search (Active-iNAS) based on data complexity change, which improves the accuracy and learning efficiency. However, Active-iNAS trains several models and selects the model with the best generalization performance for querying the subsequent samples after each active learning cycle. The independent training processes lead to an insufferable computational budget, which is significantly inefficient and limits search flexibility and final performance. To address this issue, we propose a novel active strategy with the method called structured variational inference (SVI) or structured neural depth search (SNDS) whereby we could use the gradient descent method in neural network depth search during AL processes. At the same time, we theoretically demonstrate that the current VI-based methods based on the mean-field assumption could lead to poor performance. We apply our strategy using three querying techniques and three datasets and show that our strategy outperforms current methods.

翻译：先前工作通过基于数据复杂度变化的自适应神经网络架构搜索（Active-iNAS）来优化传统主动学习（AL）流程，从而提升了准确率与学习效率。然而，Active-iNAS在每个主动学习周期后需训练多个模型，并选择泛化性能最优的模型用于后续样本查询。这种独立训练过程导致了难以承受的计算开销，严重降低效率，同时限制了搜索灵活性与最终性能。为解决此问题，我们提出一种新型主动策略，其核心方法称为结构化变分推断（SVI）或结构化神经深度搜索（SNDS），该方法允许我们在主动学习过程中采用梯度下降法进行神经网络深度搜索。同时，我们从理论上证明：当前基于均值场假设的变分推断方法会导致性能劣化。我们采用三种查询技术与三个数据集对所提策略进行验证，结果表明该方法显著优于现有方案。