Golden-section search and bisection search are the two main principled algorithms for 1d minimization of quasiconvex (unimodal) functions. The first one only uses function queries, while the second one also uses gradient queries. Other algorithms exist under much stronger assumptions, such as Newton's method. However, to the best of our knowledge, there is no principled exact line search algorithm for general convex functions -- including piecewise-linear and max-compositions of convex functions -- that takes advantage of convexity. We propose two such algorithms: $\Delta$-Bisection is a variant of bisection search that uses (sub)gradient information and convexity to speed up convergence, while $\Delta$-Secant is a variant of golden-section search and uses only function queries. While bisection search reduces the $x$ interval by a factor 2 at every iteration, $\Delta$-Bisection reduces the (sometimes much) smaller $x^*$-gap $\Delta^x$ (the $x$ coordinates of $\Delta$) by at least a factor 2 at every iteration. Similarly, $\Delta$-Secant also reduces the $x^*$-gap by at least a factor 2 every second function query. Moreover, the $y^*$-gap $\Delta^y$ (the $y$ coordinates of $\Delta$) also provides a refined stopping criterion, which can also be used with other algorithms. Experiments on a few convex functions confirm that our algorithms are always faster than their quasiconvex counterparts, often by more than a factor 2. We further design a quasi-exact line search algorithm based on $\Delta$-Secant. It can be used with gradient descent as a replacement for backtracking line search, for which some parameters can be finicky to tune -- and we provide examples to this effect, on strongly-convex and smooth functions. We provide convergence guarantees, and confirm the efficiency of quasi-exact line search on a few single- and multivariate convex functions.

翻译：黄金分割搜索和二分搜索是拟凸（单峰）函数一维极小化的两种主要原则性算法。前者仅使用函数查询，后者则额外使用梯度查询。在更强假设下存在其他算法，如牛顿法。然而，据我们所知，目前尚无利用凸性的一般凸函数——包括分段线性凸函数及凸函数的极大复合——的原则性精确线搜索算法。我们提出两种此类算法：Δ-二分法是二分搜索的变体，利用（次）梯度信息和凸性加速收敛；Δ-割线法是黄金分割搜索的变体，仅使用函数查询。二分搜索每轮迭代将$x$区间缩减一半，而Δ-二分法每轮迭代将（通常更小的）$x^*$-间隙$\Delta^x$（$\Delta$的$x$坐标）至少缩减一半。类似地，Δ-割线法每两次函数查询将$x^*$-间隙至少缩减一半。此外，$y^*$-间隙$\Delta^y$（$\Delta$的$y$坐标）提供了更精细的停止准则，也可用于其他算法。在若干凸函数上的实验证实，我们的算法始终快于其拟凸对应算法，速度提升通常超过一倍。我们进一步设计了基于Δ-割线法的准精确线搜索算法。该算法可替代梯度下降法中的回溯线搜索，避免了后者某些参数难以调优的问题——我们给出了强凸和光滑函数上的相关示例。我们提供了收敛性保证，并在若干单变量和多变量凸函数上验证了准精确线搜索的效率。