Algorithmic stability is a central concept in statistics and learning theory that measures how sensitive an algorithm's output is to small changes in the training data. Stability plays a crucial role in understanding generalization, robustness, and replicability, and a variety of stability notions have been proposed in different learning settings. However, while stability entails desirable properties, it is typically not sufficient on its own for statistical learning -- and indeed, it may be at odds with accuracy, since an algorithm that always outputs a constant function is perfectly stable but statistically meaningless. Thus, it is essential to understand the potential statistical cost of stability. In this work, we address this question by adopting a statistical decision-theoretic perspective, treating stability as a constraint in estimation. Focusing on two representative notions-worst-case stability and average-case stability-we first establish general lower bounds on the achievable estimation accuracy under each type of stability constraint. We then develop optimal stable estimators for four canonical estimation problems, including several mean estimation and regression settings. Together, these results characterize the optimal trade-offs between stability and accuracy across these tasks. Our findings formalize the intuition that average-case stability imposes a qualitatively weaker restriction than worst-case stability, and they further reveal that the gap between these two can vary substantially across different estimation problems.

翻译：算法稳定性是统计学与学习理论中的核心概念，它衡量算法输出对训练数据微小变化的敏感程度。稳定性在理解泛化性、鲁棒性和可复现性方面起着关键作用，且在不同学习场景中已提出多种稳定性概念。然而，尽管稳定性蕴含了理想的性质，其本身通常不足以满足统计学习的要求——事实上，稳定性可能与准确性相冲突，因为始终输出常数函数的算法虽完全稳定，却缺乏统计意义。因此，理解稳定性可能带来的统计代价至关重要。本研究通过采用统计决策理论的视角，将稳定性视为估计中的约束条件，以探讨这一问题。聚焦于两种代表性概念——最坏情况稳定性与平均情况稳定性——我们首先建立了在每种稳定性约束下可达到的估计精度的一般下界。随后，我们针对四个经典估计问题（包括若干均值估计与回归场景）构建了最优的稳定估计器。这些结果共同刻画了在这些任务中稳定性与准确性之间的最优权衡关系。我们的发现形式化了以下直观认识：平均情况稳定性所施加的限制在性质上弱于最坏情况稳定性，并进一步揭示这两种稳定性之间的差距在不同估计问题中可能存在显著差异。