Under losses which are potentially heavy-tailed, we consider the task of minimizing sums of the loss mean and standard deviation, without trying to accurately estimate the variance. By modifying a technique for variance-free robust mean estimation to fit our problem setting, we derive a simple learning procedure which can be easily combined with standard gradient-based solvers to be used in traditional machine learning workflows. Empirically, we verify that our proposed approach, despite its simplicity, performs as well or better than even the best-performing candidates derived from alternative criteria such as CVaR or DRO risks on a variety of datasets.
翻译:在损失可能具有重尾特性的情况下,我们考虑最小化损失均值与标准差之和的任务,而无需精确估计方差。通过将一种无方差的鲁棒均值估计技术适配到我们的问题设定中,我们推导出一种简单的学习程序,该程序可轻松与标准梯度求解器结合,用于传统机器学习工作流程。实验验证表明,我们提出的方法尽管简单,但在多种数据集上的表现与基于CVaR或DRO风险等替代准则的最优候选方法相当甚至更优。