Preference elicitation leverages AI or optimization to learn stakeholder preferences in settings ranging from marketing to public policy. The online robust preference elicitation procedure of arXiv:2003.01899 has been shown in simulation to outperform various other elicitation procedures in terms of effectively learning individuals' true utilities. However, as with any simulation, the method makes a series of assumptions that cannot easily be verified to hold true beyond simulation. Thus, we propose to validate the robust method's performance in deployment, focused on the particular challenge of selecting policies for prioritizing COVID-19 patients for scarce hospital resources during the pandemic. To this end, we develop an online platform for preference elicitation where users report their preferences between alternatives over a moderate number of pairwise comparisons chosen by a particular elicitation procedure. We recruit Amazon Mechanical Turk workers ($n$ = 193) to report their preferences and demonstrate that the robust method outperforms asking random queries by 21%, the next best performing method in the simulated results of arXiv:2003.01899, in terms of recommending policies with a higher utility.

翻译：偏好获取利用人工智能或优化技术，从市场营销到公共政策等场景中学习利益相关者的偏好。arXiv:2003.01899提出的在线鲁棒偏好获取方法，在模拟实验中已被证明能更有效地学习个体真实效用，优于其他多种获取方法。然而，如同任何模拟实验，该方法依赖一系列假设，这些假设在超出模拟环境后难以验证其有效性。因此，我们提出在真实部署场景中验证该鲁棒方法的性能，重点关注疫情期间稀缺医院资源中COVID-19患者优先级排序策略的选择这一特定挑战。为此，我们开发了一个在线偏好获取平台，用户通过由特定获取方法选择的适中数量的成对比较，报告自己对不同选项的偏好。我们招募了亚马逊土耳其机器人工作者（$n$ = 193）来报告其偏好，结果表明：在推荐更高效用策略方面，该鲁棒方法比随机提问方法（即arXiv:2003.01899模拟结果中表现次优的方法）提升了21%。