The increasing reliance on human preference feedback to judge AI-generated pseudo labels has created a pressing need for principled, budget-conscious data acquisition strategies. We address the crucial question of how to optimally allocate a fixed annotation budget between ground-truth labels and pairwise preferences in AI. Our solution, grounded in semi-parametric inference, casts the budget allocation problem as a monotone missing data framework. Building on this formulation, we introduce Preference-Calibrated Active Learning (PCAL), a novel method that learns the optimal data acquisition strategy and develops a statistically efficient estimator for functionals of the data distribution. Theoretically, we prove the asymptotic optimality of our PCAL estimator and establish a key robustness guarantee that ensures robust performance even with poorly estimated nuisance models. Our flexible framework applies to a general class of problems, by directly optimizing the estimator's variance instead of requiring a closed-form solution. This work provides a principled and statistically efficient approach for budget-constrained learning in modern AI. Simulations and real-data analysis demonstrate the practical benefits and superior performance of our proposed method.

翻译：随着人类偏好反馈在评判AI生成的伪标签方面日益受到依赖，制定原则性且预算敏感的数据获取策略变得尤为迫切。本文探讨了一个关键问题：在人工智能领域，如何将固定的标注预算最优地分配给真实标签和成对偏好。我们的解决方案基于半参数推断，将预算分配问题构建为一个单调缺失数据框架。基于此框架，我们提出了偏好校准主动学习（PCAL），这是一种新颖的方法，既能学习最优的数据获取策略，又能为数据分布的函数构建统计高效的估计量。理论上，我们证明了PCAL估计量的渐近最优性，并建立了一个关键的鲁棒性保证，确保即使在辅助模型估计不佳的情况下也能保持稳健性能。我们的灵活框架适用于一大类问题，它通过直接优化估计量的方差，而非要求闭式解来实现。这项工作为现代人工智能中的预算约束学习提供了一种原则性且统计高效的方法。仿真和实际数据分析证明了我们所提方法的实用优势与优越性能。