We prove impossibility results for adaptivity in non-smooth stochastic convex optimization. Given a set of problem parameters we wish to adapt to, we define a "price of adaptivity" (PoA) that, roughly speaking, measures the multiplicative increase in suboptimality due to uncertainty in these parameters. When the initial distance to the optimum is unknown but a gradient norm bound is known, we show that the PoA is at least logarithmic for expected suboptimality, and double-logarithmic for median suboptimality. When there is uncertainty in both distance and gradient norm, we show that the PoA must be polynomial in the level of uncertainty. Our lower bounds nearly match existing upper bounds, and establish that there is no parameter-free lunch. En route, we also establish tight upper and lower bounds for (known-parameter) high-probability stochastic convex optimization with heavy-tailed and bounded noise, respectively.

翻译：我们证明了非光滑随机凸优化中自适应性的不可能性结果。针对一组我们希望自适应的参数，我们定义了"自适应代价"，粗略地说，它衡量了由于这些参数的不确定性所导致的次优性的倍增程度。当初始点到最优点的距离未知但梯度范数界已知时，我们证明了对于期望次优性，自适应代价至少是对数级别的；对于中位数次优性，则是双对数级别的。当距离和梯度范数都存在不确定性时，我们证明了自适应代价必须是不确定性水平的多项式级别。我们的下界几乎匹配了现有的上界，并确立了不存在无参数的免费午餐。在此过程中，我们还分别针对具有重尾噪声和有界噪声的（已知参数）高概率随机凸优化，建立了紧的上界和下界。