A fundamental problem in statistics and machine learning is to estimate a function $f$ from possibly noisy observations of its point samples. The goal is to design a numerical algorithm to construct an approximation $\hat f$ to $f$ in a prescribed norm that asymptotically achieves the best possible error (as a function of the number $m$ of observations and the variance $\sigma^2$ of the noise). This problem has received considerable attention in both nonparametric statistics (noisy observations) and optimal recovery (noiseless observations). Quantitative bounds require assumptions on $f$, known as model class assumptions. Classical results assume that $f$ is in the unit ball of a Besov space. In nonparametric statistics, the best possible performance of an algorithm for finding $\hat f$ is known as the minimax rate and has been studied in this setting under the assumption that the noise is Gaussian. In optimal recovery, the best possible performance of an algorithm is known as the optimal recovery rate and has also been determined in this setting. While one would expect that the minimax rate recovers the optimal recovery rate when the noise level $\sigma$ tends to zero, it turns out that the current results on minimax rates do not carefully determine the dependence on $\sigma$ and the limit cannot be taken. This paper handles this issue and determines the noise-level-aware (NLA) minimax rates for Besov classes when error is measured in an $L_q$-norm with matching upper and lower bounds. The end result is a reconciliation between minimax rates and optimal recovery rates. The NLA minimax rate continuously depends on the noise level and recovers the optimal recovery rate when $\sigma$ tends to zero.
翻译:统计学与机器学习中的一个基本问题是从可能含有噪声的点样本观测中估计函数$f$。目标在于设计一种数值算法,以在指定范数下构造$f$的近似$\hat f$,使其渐近地达到可能的最佳误差(作为观测次数$m$和噪声方差$\sigma^2$的函数)。该问题在非参数统计学(含噪声观测)和最优恢复(无噪声观测)领域均受到广泛关注。定量界需要关于$f$的假设,即模型类假设。经典结果假定$f$位于Besov空间的单位球内。在非参数统计学中,寻找$\hat f$的算法所能达到的最佳性能被称为极小极大速率,并在假设噪声为高斯噪声的条件下于该设定中得到研究。在最优恢复中,算法的最佳性能被称为最优恢复速率,同样在此设定下得以确定。尽管人们预期当噪声水平$\sigma$趋于零时极小极大速率会恢复最优恢复速率,但现有的极小极大速率结果并未细致确定其对$\sigma$的依赖关系,因此无法直接取极限。本文处理了这一问题,在误差以$L_q$范数度量时,确定了Besov类的噪声水平感知(NLA)极小极大速率,并给出了匹配的上界与下界。最终结果实现了极小极大速率与最优恢复速率的统一。NLA极小极大速率连续依赖于噪声水平,并在$\sigma$趋于零时恢复最优恢复速率。