We study the problem of global optimization, where we analyze the performance of the Piyavskii--Shubert algorithm and its variants. For any given time duration $T$, instead of the extensively studied simple regret (which is the difference of the losses between the best estimate up to $T$ and the global minimum), we study the cumulative regret up to time $T$. For $L$-Lipschitz continuous functions, we show that the cumulative regret is $O(L\log T)$. For $H$-Lipschitz smooth functions, we show that the cumulative regret is $O(H)$. We analytically extend our results for functions with Holder continuous derivatives, which cover both the Lipschitz continuous and the Lipschitz smooth functions, individually. We further show that a simpler variant of the Piyavskii-Shubert algorithm performs just as well as the traditional variants for the Lipschitz continuous or the Lipschitz smooth functions. We further extend our results to broader classes of functions, and show that, our algorithm efficiently determines its queries; and achieves nearly minimax optimal (up to log factors) cumulative regret, for general convex or even concave regularity conditions on the extrema of the objective (which encompasses many preceding regularities). We consider further extensions by investigating the performance of the Piyavskii-Shubert variants in the scenarios with unknown regularity, noisy evaluation and multivariate domain.
翻译:我们研究全局优化问题,分析Piyavskii-Shubert算法及其变体的性能。对于任意给定的时间长度$T$,不同于被广泛研究的简单遗憾(即截至时间$T$的最佳估计值与全局最小值之间的损失差),我们研究截至时间$T$的累积遗憾。对于$L$-Lipschitz连续函数,我们证明累积遗憾为$O(L\log T)$。对于$H$-Lipschitz光滑函数,我们证明累积遗憾为$O(H)$。我们通过解析方法将结果推广至具有Hölder连续导数的函数,该函数类分别涵盖Lipschitz连续函数和Lipschitz光滑函数。进一步,我们证明Piyavskii-Shubert算法的一个更简单变体在Lipschitz连续或Lipschitz光滑函数上的表现与传统变体同样有效。我们还将结果推广至更广泛的函数类,并表明算法能够高效确定查询点;对于目标函数极值点满足一般凸或凹正则性条件(涵盖多种先前正则性条件)的情形,该算法实现几乎极小极大最优(仅含对数因子)的累积遗憾。我们进一步考虑扩展研究,包括在未知正则性、带噪声评估以及多变量域场景中分析Piyavskii-Shubert变体的性能。