We provide non-asymptotic excess risk guarantees for statistical learning in a setting where the population risk with respect to which we evaluate the target parameter depends on an unknown nuisance parameter that must be estimated from data. We analyze a two-stage sample splitting meta-algorithm that takes as input arbitrary estimation algorithms for the target parameter and nuisance parameter. We show that if the population risk satisfies a condition called Neyman orthogonality, the impact of the nuisance estimation error on the excess risk bound achieved by the meta-algorithm is of second order. Our theorem is agnostic to the particular algorithms used for the target and nuisance and only makes an assumption on their individual performance. This enables the use of a plethora of existing results from machine learning to give new guarantees for learning with a nuisance component. Moreover, by focusing on excess risk rather than parameter estimation, we can provide rates under weaker assumptions than in previous works and accommodate settings in which the target parameter belongs to a complex nonparametric class. We provide conditions on the metric entropy of the nuisance and target classes such that oracle rates of the same order as if we knew the nuisance parameter are achieved.

翻译：本文针对一种统计学习场景提供了非渐近超额风险保证：在该场景中，用于评估目标参数的总体风险依赖于一个必须从数据中估计的未知干扰参数。我们分析了一个两阶段样本分割元算法，该算法以目标参数和干扰参数的任意估计算法作为输入。我们证明，如果总体风险满足奈曼正交性条件，则干扰参数的估计误差对元算法实现的超额风险界的影响是二阶的。我们的定理对目标参数和干扰参数所使用的具体算法保持无偏性，仅假设各自算法的个体性能。这使得我们可以利用机器学习领域的大量现有成果，为含干扰组件的学习问题提供新的理论保证。此外，通过聚焦超额风险而非参数估计，我们能在比以往研究更弱的假设条件下给出收敛速率，并容纳目标参数属于复杂非参数类的情况。我们给出了干扰参数类与目标参数类的度量熵条件，使得在不知道干扰参数的情况下仍能实现与已知干扰参数情形同阶的oracle收敛速率。