A crucial assumption underlying the most current theory of machine learning is that the training distribution is identical to the test distribution. However, this assumption may not hold in some real-world applications. In this paper, we develop a learning model based on principles of information theory by minimizing the worst-case loss at prescribed levels of uncertainty. We reformulate the empirical estimation of the risk functional and the distribution deviation constraint based on the importance sampling method. The objective of the proposed approach is to minimize the loss under maximum degradation and hence the resulting problem is a minimax problem which can be converted to an unconstrained minimum problem using the Lagrange method with the Lagrange multiplier $T$. We reveal that the minimization of the objective function under logarithmic transformation is equivalent to the minimization of the p-norm loss with $p=\frac{1}{T}$. We applied the proposed model to the face verification task on Racial Faces in the Wild datasets and showed that the proposed model performs better under large distribution deviations.

翻译：当前机器学习理论中最基本的假设之一是训练分布与测试分布相同。然而，这一假设在某些实际应用中可能不成立。本文基于信息论原理，通过最小化指定不确定性水平下的最坏情况损失，提出了一种学习模型。我们利用重要性采样方法重新表述了风险泛函的经验估计及分布偏差约束。所提方法的目标是在最大退化条件下最小化损失，因此该问题可转化为一个极小极大问题，并通过拉格朗日乘子 $T$ 的拉格朗日方法将其转换为无约束最小化问题。我们揭示了在对数变换下目标函数的最小化等价于 p-范数损失（其中 $p=\frac{1}{T}$）的最小化。我们将所提模型应用于Racial Faces in the Wild数据集上的人脸验证任务，结果表明在较大分布偏差下该模型性能更优。