We propose Active Surrogate Estimators (ASEs), a new method for label-efficient model evaluation. Evaluating model performance is a challenging and important problem when labels are expensive. ASEs address this active testing problem using a surrogate-based estimation approach, whereas previous methods have focused on Monte Carlo estimates. ASEs actively learn the underlying surrogate, and we propose a novel acquisition strategy, XWING, that tailors this learning to the final estimation task. We find that ASEs offer greater label-efficiency than the current state-of-the-art when applied to challenging model evaluation problems for deep neural networks. We further theoretically analyze ASEs' errors.

翻译：我们建议使用主动代用模拟器(SASEs),这是一种新的标签高效模型评估方法。在标签昂贵的情况下,评估模型性能是一个具有挑战性和重要的问题。 使用代用估算法解决这个积极的测试问题,而以前的方法侧重于蒙特卡洛的估计。 个体经济实体积极学习了潜在的代用模型,我们提出了一种新的购置战略,即XWING,使这一学习适应最终的估算任务。我们发现,在应用AESE来应对深神经网络的挑战性模型评估问题时,其标签效率比目前的最新水平要高。我们进一步从理论上分析个体经济实体的错误。