We study the excess minimum risk in statistical inference, defined as the difference between the minimum expected loss in estimating a random variable from an observed feature vector and the minimum expected loss in estimating the same random variable from a transformation (statistic) of the feature vector. After characterizing lossless transformations, i.e., transformations for which the excess risk is zero for all loss functions, we construct a partitioning test statistic for the hypothesis that a given transformation is lossless and show that for i.i.d. data the test is strongly consistent. More generally, we develop information-theoretic upper bounds on the excess risk that uniformly hold over fairly general classes of loss functions. Based on these bounds, we introduce the notion of a delta-lossless transformation and give sufficient conditions for a given transformation to be universally delta-lossless. Applications to classification, nonparametric regression, portfolio strategies, information bottleneck, and deep learning, are also surveyed.

翻译：我们研究统计推断中的超额最小风险，定义为根据观测特征向量估计随机变量的最小期望损失与根据特征向量的变换（统计量）估计同一随机变量的最小期望损失之差。在刻画了无损变换（即对所有损失函数超额风险为零的变换）后，我们构建了一个划分检验统计量用于检验给定变换是否为无损变换，并证明对于独立同分布数据该检验具有强相合性。更一般地，我们建立了超额风险的信息论上界，该上界在相当广泛的损失函数类上一致成立。基于这些界，我们引入了δ-无损变换的概念，并给出了给定变换成为通用δ-无损变换的充分条件。本文还综述了该方法在分类、非参数回归、投资组合策略、信息瓶颈和深度学习中的应用。