Training machine learning and statistical models often involves optimizing a data-driven risk criterion. The risk is usually computed with respect to the empirical data distribution, but this may result in poor and unstable out-of-sample performance due to distributional uncertainty. In the spirit of distributionally robust optimization, we propose a novel robust criterion by combining insights from Bayesian nonparametric (i.e., Dirichlet Process) theory and recent decision-theoretic models of smooth ambiguity-averse preferences. First, we highlight novel connections with standard regularized empirical risk minimization techniques, among which Ridge and LASSO regressions. Then, we theoretically demonstrate the existence of favorable finite-sample and asymptotic statistical guarantees on the performance of the robust optimization procedure. For practical implementation, we propose and study tractable approximations of the criterion based on well-known Dirichlet Process representations. We also show that the smoothness of the criterion naturally leads to standard gradient-based numerical optimization. Finally, we provide insights into the workings of our method by applying it to high-dimensional sparse linear regression and robust location parameter estimation tasks.

翻译：训练机器学习与统计模型通常涉及优化数据驱动的风险准则。该风险通常基于经验数据分布计算，但由于分布不确定性，可能导致样本外性能欠佳且不稳定。受分布鲁棒优化的启发，我们通过融合贝叶斯非参数（即狄利克雷过程）理论与近期平滑模糊厌恶偏好的决策理论模型，提出了一种新颖的鲁棒准则。首先，我们揭示了该方法与标准正则化经验风险最小化技术（包括岭回归和LASSO回归）之间的新联系。随后，我们从理论上证明了该鲁棒优化过程在有限样本和渐近统计性能上存在有利保证。在实际实现中，我们基于众所周知的狄利克雷过程表示，提出并研究了该准则的可处理近似方法。同时，我们展示了该准则的平滑性自然导出了基于梯度的标准数值优化方法。最后，通过高维稀疏线性回归和鲁棒位置参数估计任务的应用，我们深入阐释了该方法的工作原理。