Performative predictions influence the very outcomes they aim to forecast. We study performative predictions that affect a sample (e.g., only existing users of an app) and/or the whole population (e.g., all potential app users). This raises the question of how well models generalize under performativity. For example, how well can we draw insights about new app users based on existing users when both of them react to the app's predictions? We address this question by embedding performative predictions into statistical learning theory. We prove generalization bounds under performative effects on the sample, on the population, and on both. A key intuition behind our proofs is that in the worst case, the population negates predictions, while the sample deceptively fulfills them. We cast such self-negating and self-fulfilling predictions as min-max and min-min risk functionals in Wasserstein space, respectively. Our analysis reveals a fundamental trade-off between performatively changing the world and learning from it: the more a model affects data, the less it can learn from it. Moreover, our analysis results in a surprising insight on how to improve generalization guarantees by retraining on performatively distorted samples. We illustrate our bounds in a case study on prediction-informed assignments of unemployed German residents to job trainings, drawing upon administrative labor market records from 1975 to 2017 in Germany.
翻译:表演性预测会影响它们试图预测的结果本身。我们研究影响样本(例如,仅限已有的应用程序用户)和/或整个群体(例如,所有潜在的应用程序用户)的表演性预测。这提出了一个问题:在表演性条件下,模型的泛化能力如何。例如,当新旧应用用户都对应用的预测做出反应时,我们基于现有用户对新应用用户的研究结论能在多大程度上保持有效?我们通过将表演性预测嵌入统计学习理论来解决这个问题。我们证明了在样本、群体以及两者同时受到表演性影响时的泛化界。我们证明背后的一个关键直觉是:在最坏情况下,群体否定预测,而样本则欺骗性地实现预测。我们将这种自我否定和自我实现的预测分别表述为Wasserstein空间中的极小极大和极小极小风险泛函。我们的分析揭示了表演性改变世界与向世界学习之间的一个基本权衡:一个模型对数据的影响越大,它从数据中学到的就越少。此外,我们的分析得出了一个令人惊讶的洞见,即通过在使用表演性扭曲的样本上进行重新训练,可以如何改进泛化保证。我们以一项基于1975年至2017年德国行政劳动力市场记录的案例研究来说明我们的界,该案例涉及根据预测将德国失业居民分配至职业培训。