Many novel notions of "risk" (e.g., CVaR, tilted risk, DRO risk) have been proposed and studied, but these risks are all at least as sensitive as the mean to loss tails on the upside, and tend to ignore deviations on the downside. We study a complementary new risk class that penalizes loss deviations in a bi-directional manner, while having more flexibility in terms of tail sensitivity than is offered by mean-variance. This class lets us derive high-probability learning guarantees without explicit gradient clipping, and empirical tests using both simulated and real data illustrate a high degree of control over key properties of the test loss distribution incurred by gradient-based learners.
翻译:许多新颖的“风险”概念(例如条件风险价值、倾斜风险、分布鲁棒优化风险)已被提出并深入研究,但这些风险对损失上尾部的敏感度至少不低于均值,且往往忽略下尾部的偏差。我们研究了一个互补的新型风险类,它以双向方式惩罚损失偏离,同时在尾部敏感度上比均值-方差框架具有更高的灵活性。该风险类使我们无需显式梯度裁剪即可推导出高概率学习保证,且基于模拟数据与真实数据的实证测试表明,其对基于梯度的学习器所生成的测试损失分布的关键属性具有高度可控性。