We study the problem of estimating the derivatives of a regression function, which has a wide range of applications as a key nonparametric functional of unknown functions. Standard analysis may be tailored to specific derivative orders, and parameter tuning remains a daunting challenge particularly for high-order derivatives. In this article, we propose a simple plug-in kernel ridge regression (KRR) estimator in nonparametric regression with random design that is broadly applicable for multi-dimensional support and arbitrary mixed-partial derivatives. We provide a non-asymptotic analysis to study the behavior of the proposed estimator in a unified manner that encompasses the regression function and its derivatives, leading to two error bounds for a general class of kernels under the strong $L_\infty$ norm. In a concrete example specialized to kernels with polynomially decaying eigenvalues, the proposed estimator recovers the minimax optimal rate up to a logarithmic factor for estimating derivatives of functions in H\"older and Sobolev classes. Interestingly, the proposed estimator achieves the optimal rate of convergence with the same choice of tuning parameter for any order of derivatives. Hence, the proposed estimator enjoys a \textit{plug-in property} for derivatives in that it automatically adapts to the order of derivatives to be estimated, enabling easy tuning in practice. Our simulation studies show favorable finite sample performance of the proposed method relative to several existing methods and corroborate the theoretical findings on its minimax optimality.

翻译：本文研究回归函数导数的估计问题，该问题作为未知函数的关键非参数泛函具有广泛应用。标准分析方法可能针对特定导数阶数进行定制，而参数调优尤其对于高阶导数仍是严峻挑战。本文针对随机设计下的非参数回归，提出一种简洁的插件核岭回归（KRR）估计量，该估计量广泛适用于多维支撑和任意混合偏导数。我们通过统一框架给出非渐近分析以研究该估计量的行为（涵盖回归函数及其导数），针对一般核函数类在强$L_\infty$范数下导出两个误差界。在以多项式衰减特征值为特征的核函数具体案例中，所提估计量在估计Hölder和Sobolev类函数导数时，恢复达到最小化最优速率（仅相差对数因子）。值得关注的是，该估计量对任意阶导数均可通过相同调优参数选择达到最优收敛速率。因此，所提估计量对导数具有**插件性质**——能自动适应待估导数阶数，从而在实践应用中实现简易调参。仿真研究表明，相对于现有多种方法，本方法在有限样本下表现优异，并验证了其最小化最优性的理论结果。