Under losses which are potentially heavy-tailed, we consider the task of minimizing sums of the loss mean and standard deviation, without trying to accurately estimate the variance. By modifying a technique for variance-free robust mean estimation to fit our problem setting, we derive a simple learning procedure which can be easily combined with standard gradient-based solvers to be used in traditional machine learning workflows. Empirically, we verify that our proposed approach, despite its simplicity, performs as well or better than even the best-performing candidates derived from alternative criteria such as CVaR or DRO risks on a variety of datasets.
翻译:在可能具有重尾分布的损失函数下,我们考虑最小化损失均值与标准差之和的任务,而无需精确估计方差。通过将一种无方差鲁棒均值估计技术适配至我们的问题设定,我们推导出一个简单的学习流程,该流程可轻松与标准基于梯度的求解器结合,用于传统机器学习工作流。实验验证表明,尽管方法简洁,但在多种数据集上,我们所提方法的性能与基于CVaR或DRO风险等替代准则的最优候选方法相当甚至更优。