Prediction algorithms are increasingly used to inform decisions about humans, but maximizing accuracy$\rule[0.25em]{1em}{0.4pt}$the standard learning objective$\rule[0.25em]{1em}{0.4pt}$does not necessarily maximize user benefits. Instead, we propose optimizing social welfare, defined as the average gain users receive from correct predictions. Welfare enables to express, and therefore account for, heterogeneity in how much users benefit from accuracy. But since these valuations are private and users can gain from overreporting them, learning must simultaneously elicit truthful values and optimize welfare with respect to them. To this end, we propose a novel learning algorithm that incorporates a truthful auction. We show how to compute allocations and prices efficiently, and bound the number of paying users$\rule[0.25em]{1em}{0.4pt}$ which surprisingly is independent of the sample size. We conclude with experiments on real and synthetic data that demonstrate our algorithm and explore the connections between welfare and accuracy.
翻译:预测算法日益被用于影响人类决策,但最大化准确度——这一标准学习目标——并不总能最大化用户收益。为此,我们提出优化社会福利,定义为用户因正确预测获得的平均收益。社会福利能够表达并因此考虑用户从准确度中获益的异质性。然而,由于这些估值是私密的且用户可能通过虚报获益,学习过程必须同时激励真实报告估值并优化相应的社会福利。为此,我们提出一种融合真实拍卖的新型学习算法。我们展示了如何有效计算分配与价格,并约束付费用户数量——这一数量出人意料地与样本容量无关。最后,通过真实与合成数据实验验证算法性能,并探讨社会福利与准确度之间的关联。