It is a common phenomenon that for high-dimensional and nonparametric statistical models, rate-optimal estimators balance squared bias and variance. Although this balancing is widely observed, little is known whether methods exist that could avoid the trade-off between bias and variance. We propose a general strategy to obtain lower bounds on the variance of any estimator with bias smaller than a prespecified bound. This shows to which extent the bias-variance trade-off is unavoidable and allows to quantify the loss of performance for methods that do not obey it. The approach is based on a number of abstract lower bounds for the variance involving the change of expectation with respect to different probability measures as well as information measures such as the Kullback-Leibler or $\chi^2$-divergence. In a second part of the article, the abstract lower bounds are applied to several statistical models including the Gaussian white noise model, a boundary estimation problem, the Gaussian sequence model and the high-dimensional linear regression model. For these specific statistical applications, different types of bias-variance trade-offs occur that vary considerably in their strength. For the trade-off between integrated squared bias and integrated variance in the Gaussian white noise model, we propose to combine the general strategy for lower bounds with a reduction technique. This allows us to reduce the original problem to a lower bound on the bias-variance trade-off for estimators with additional symmetry properties in a simpler statistical model. In the Gaussian sequence model, different phase transitions of the bias-variance trade-off occur. Although there is a non-trivial interplay between bias and variance, the rate of the squared bias and the variance do not have to be balanced in order to achieve the minimax estimation rate.

翻译：在高维和非参数统计模型中，速率最优估计器平衡平方偏差与方差是常见现象。尽管这种平衡被广泛观测到，但关于是否存在能够避免偏差-方差权衡的方法却鲜为人知。我们提出了一种通用策略，用于推导任何偏差小于预设下界的估计器方差的下界。这揭示了偏差-方差权衡在何种程度上不可避免，并允许量化未遵循该权衡的方法的性能损失。该方法基于若干涉及不同概率测度下期望变化以及信息测度（如Kullback-Leibler散度或$\chi^2$-散度）的方差抽象下界。在文章的第二部分，我们将这些抽象下界应用于多个统计模型，包括高斯白噪声模型、边界估计问题、高斯序列模型和高维线性回归模型。针对这些具体的统计应用，出现了强度各异的多种偏差-方差权衡类型。对于高斯白噪声模型中的积分平方偏差与积分方差权衡，我们提出将通用下界策略与降维技术相结合，从而将原始问题简化为在更简单统计模型中具有额外对称性的估计器偏差-方差权衡的下界。在高斯序列模型中，偏差-方差权衡表现出不同的相变现象。尽管偏差与方差之间存在非平凡相互作用，但实现最小最大估计速率并不需要平方偏差与方差的速率保持平衡。