Superquantiles have recently gained significant interest as a risk-aware metric for addressing fairness and distribution shifts in statistical learning and decision making problems. This paper introduces a fast, scalable and robust second-order computational framework to solve large-scale optimization problems with superquantile-based constraints. Unlike empirical risk minimization, superquantile-based optimization requires ranking random functions evaluated across all scenarios to compute the tail conditional expectation. While this tail-based feature might seem computationally unfriendly, it provides an advantageous setting for a semismooth-Newton-based augmented Lagrangian method. The superquantile operator effectively reduces the dimensions of the Newton systems since the tail expectation involves considerably fewer scenarios. Notably, the extra cost of obtaining relevant second-order information and performing matrix inversions is often comparable to, and sometimes even less than, the effort required for gradient computation. Our developed solver is particularly effective when the number of scenarios substantially exceeds the number of decision variables. In synthetic problems with linear and convex diagonal quadratic objectives, numerical experiments demonstrate that our method outperforms existing approaches by a large margin: It achieves speeds more than 750 times faster for linear and quadratic objectives than the alternating direction method of multipliers as implemented by OSQP for computing low-accuracy solutions. Additionally, it is up to 25 times faster for linear objectives and 70 times faster for quadratic objectives than the commercial solver Gurobi, and 20 times faster for linear objectives and 30 times faster for quadratic objectives than the Portfolio Safeguard optimization suite for high-accuracy solution computations.

翻译：超分位数最近作为一种风险感知指标，在处理统计学习与决策问题中的公平性和分布偏移方面引起了广泛兴趣。本文提出了一种快速、可扩展且稳健的二阶计算框架，用于求解具有超分位数约束的大规模优化问题。与经验风险最小化不同，基于超分位数的优化需要对所有场景下评估的随机函数进行排序，以计算尾部条件期望。尽管这种基于尾部的特性看似对计算不友好，但它为基于半光滑牛顿法的增广拉格朗日方法提供了有利条件。超分位数算子有效降低了牛顿系统的维度，因为尾部期望涉及的场景数量显著减少。值得注意的是，获取相关二阶信息和执行矩阵求逆的额外成本通常与梯度计算所需的工作量相当，有时甚至更少。我们开发的求解器在场景数量远超决策变量数量的情况下尤为有效。在线性和凸对角二次目标的合成问题中，数值实验表明，我们的方法在性能上大幅超越现有方法：与OSQP实现的交替方向乘子法相比，在计算低精度解时，本方法在线性和二次目标上的速度快了750倍以上。此外，与商业求解器Gurobi相比，在线性目标上快了25倍，在二次目标上快了70倍；与Portfolio Safeguard优化套件相比，在高精度解计算时，线性目标上快了20倍，二次目标上快了30倍。