In this paper, we are concerned with the generalization performance of non-parametric estimation for pairwise learning. Most of the existing work requires the hypothesis space to be convex or a VC-class, and the loss to be convex. However, these restrictive assumptions limit the applicability of the results in studying many popular methods, especially kernel methods and neural networks. We significantly relax these restrictive assumptions and establish a sharp oracle inequality of the empirical minimizer with a general hypothesis space for the Lipschitz continuous pairwise losses. As an example, we apply our general results to study pairwise least squares regression and derive an excess population risk bound that matches the minimax lower bound for the pointwise least squares regression. The key novelty lies in constructing a structured deep ReLU neural network to approximate the true predictor, and in designing a targeted hypothesis space composed of networks with this structure and controllable complexity. Experiments validate the effectiveness of the proposed method. This example demonstrates that the obtained general results indeed help us to explore the generalization performance on a variety of problems that cannot be handled by existing approaches.

翻译：本文关注配对学习中非参数估计的泛化性能。现有研究大多要求假设空间为凸集或VC类，且损失函数需满足凸性。然而，这些限制性假设制约了相关结论在研究诸多流行方法（特别是核方法与神经网络）时的适用性。我们显著放宽了这些限制性假设，针对Lipschitz连续配对损失函数，在一般假设空间下建立了经验最小化器的尖锐甲骨文不等式。作为示例，我们将一般性结论应用于配对最小二乘回归研究，推导出的超额总体风险界与逐点最小二乘回归的极小极大下界相匹配。其核心创新在于构建具有特定结构的深度ReLU神经网络以逼近真实预测器，并设计由该结构网络构成且复杂度可控的目标假设空间。实验验证了所提方法的有效性。此例证表明，所得一般性结论确实有助于探索现有方法无法处理的各类问题的泛化性能。