Supervised learning typically optimizes the expected value risk functional of the loss, but in many cases, we want to optimize for other risk functionals. In full-batch gradient descent, this is done by taking gradients of a risk functional of interest, such as the Conditional Value at Risk (CVaR) which ignores some quantile of extreme losses. However, deep learning must almost always use mini-batch gradient descent, and lack of unbiased estimators of various risk functionals make the right optimization procedure unclear. In this work, we introduce a meta-learning-based method of learning an interpretable mini-batch risk functional during model training, in a single shot. When optimizing for various risk functionals, the learned mini-batch risk functions lead to risk reduction of up to 10% over hand-engineered mini-batch risk functionals. Then in a setting where the right risk functional is unknown a priori, our method improves over baseline by 14% relative (~9% absolute). We analyze the learned mini-batch risk functionals at different points through training, and find that they learn a curriculum (including warm-up periods), and that their final form can be surprisingly different from the underlying risk functional that they optimize for.

翻译：监督学习通常优化损失函数的期望风险函数，但在许多情况下，我们希望对其他风险函数进行优化。在全批量梯度下降中，这是通过对感兴趣的风险函数（如条件风险价值(CVaR)）求梯度来实现的，该函数会忽略极端损失的某个分位数。然而，深度学习几乎必须使用小批量梯度下降，而各种风险函数缺乏无偏估计量导致正确的优化过程不明确。在这项工作中，我们引入了一种基于元学习的方法，在单次模型训练过程中学习一个可解释的小批量风险函数。当针对各种风险函数进行优化时，学习到的小批量风险函数相比人工设计的小批量风险函数，可实现高达10%的风险降低。在真实风险函数先验未知的场景中，我们的方法相比基线方法提升了14%（绝对提升约9%）。我们分析了训练过程中不同阶段学习到的小批量风险函数，发现它们学习到了课程学习策略（包括预热阶段），并且其最终形式可能与所优化的底层风险函数存在显著差异。