We analyze the query complexity of finding a local minimum in $t$ rounds on general graphs. More precisely, given a graph $G = (V,E)$ and oracle access to an unknown function $f : V \to \mathbb{R}$, the goal is to find a local minimum--a vertex $v$ such that $f(v) \leq f(u)$ for all $(u,v) \in E$--using at most $t$ rounds of interaction with the oracle. The query complexity is well understood on grids, but much less is known beyond. This abstract problem captures many optimization tasks, such as finding a local minimum of a loss function during neural network training. For each graph with $n$ vertices, we prove a deterministic upper bound of $O(t n^{1/t} (sΔ)^{1-1/t})$, where $s$ is the separation number and $Δ$ is the maximum degree of the graph. We complement this result with a randomized lower bound of $Ω(t n^{1/t}-t)$ that holds for any connected graph. We also find that parallel steepest descent with a warm start provides improved bounds for graphs with high separation number and bounded degree. To obtain our results, we utilized an advanced version of Gemini at various stages of our research. We discuss our experience in a methodology section.
翻译:我们分析了在一般图上通过$t$轮查询找到局部最小值的查询复杂度。具体而言,给定图$G = (V,E)$以及对未知函数$f : V \to \mathbb{R}$的预言机访问,目标是在最多使用$t$轮与预言机的交互中找到一个局部最小值——即满足对所有$(u,v) \in E$有$f(v) \leq f(u)$的顶点$v$。查询复杂度在网格图上已有充分理解,但在更一般图上的认知尚浅。该抽象问题涵盖了许多优化任务,例如在神经网络训练中寻找损失函数的局部最小值。对于每个具有$n$个顶点的图,我们证明了一个确定性上界$O(t n^{1/t} (sΔ)^{1-1/t})$,其中$s$为分离数,$Δ$为图的最大度。我们进一步给出了一个适用于任何连通图的随机下界$Ω(t n^{1/t}-t)$作为该结果的补充。我们还发现,对于具有高分离数和有界度的图,带热启动的并行最速下降法能够提供更优的界。为获得这些结果,我们在研究的各个阶段使用了高级版本的Gemini。我们将在方法论部分讨论相关经验。