The empirical risk minimization (ERM) problem with relative entropy regularization (ERM-RER) is investigated under the assumption that the reference measure is a $\sigma$-finite measure, and not necessarily a probability measure. Under this assumption, which leads to a generalization of the ERM-RER problem allowing a larger degree of flexibility for incorporating prior knowledge, numerous relevant properties are stated. Among these properties, the solution to this problem, if it exists, is shown to be a unique probability measure, mutually absolutely continuous with the reference measure. Such a solution exhibits a probably-approximately-correct guarantee for the ERM problem independently of whether the latter possesses a solution. For a fixed dataset and under a specific condition, the empirical risk is shown to be a sub-Gaussian random variable when the models are sampled from the solution to the ERM-RER problem. The generalization capabilities of the solution to the ERM-RER problem (the Gibbs algorithm) are studied via the sensitivity of the expected empirical risk to deviations from such a solution towards alternative probability measures. Finally, an interesting connection between sensitivity, generalization error, and lautum information is established.

翻译：本文研究了在参考测度为σ有限测度（不一定为概率测度）的假设下，具有相对熵正则化的经验风险最小化（ERM-RER）问题。这一假设推广了ERM-RER问题，允许更大灵活性地纳入先验知识，并由此建立了诸多相关性质。在这些性质中，若问题存在解，则解是唯一且与参考测度相互绝对连续的概率测度。该解独立于原始ERM问题是否存在解，均能为其提供"可能近似正确"的保证。对于固定数据集，在特定条件下，当模型从ERM-RER问题的解中采样时，经验风险被证明是次高斯随机变量。通过研究经验期望风险对解向其他概率测度偏移的敏感性，分析了ERM-RER问题（吉布斯算法）的解的泛化能力。最后，建立了敏感性、泛化误差与劳顿信息之间的有趣联系。