There is a gap between finding a first-order stationary point (FOSP) and a second-order stationary point (SOSP) under differential privacy constraints, and it remains unclear whether privately finding an SOSP is more challenging than finding an FOSP. Specifically, Ganesh et al. (2023) claimed that an $\alpha$-SOSP can be found with $\alpha=O(\frac{1}{n^{1/3}}+(\frac{\sqrt{d}}{n\epsilon})^{3/7})$, where $n$ is the dataset size, $d$ is the dimension, and $\epsilon$ is the differential privacy parameter. However, a recent analysis revealed an issue in their saddle point escape procedure, leading to weaker guarantees. Building on the SpiderBoost algorithm framework, we propose a new approach that uses adaptive batch sizes and incorporates the binary tree mechanism. Our method not only corrects this issue but also improves the results for privately finding an SOSP, achieving $\alpha=O(\frac{1}{n^{1/3}}+(\frac{\sqrt{d}}{n\epsilon})^{1/2})$. This improved bound matches the state-of-the-art for finding a FOSP, suggesting that privately finding an SOSP may be achievable at no additional cost.

翻译：在差分隐私约束下，寻找一阶稳定点与二阶稳定点之间存在性能差距，且目前尚不清楚差分隐私下寻找二阶稳定点是否比寻找一阶稳定点更具挑战性。具体而言，Ganesh等人（2023）曾声称能以$\alpha=O(\frac{1}{n^{1/3}}+(\frac{\sqrt{d}}{n\epsilon})^{3/7})$的精度找到$\alpha$-二阶稳定点，其中$n$为数据集规模，$d$为维度，$\epsilon$为差分隐私参数。然而，近期分析发现其鞍点逃逸过程存在缺陷，导致实际保证弱于原结论。基于SpiderBoost算法框架，我们提出一种采用自适应批量大小并结合二叉树机制的新方法。该方法不仅修正了上述缺陷，同时提升了差分隐私下寻找二阶稳定点的结果，实现了$\alpha=O(\frac{1}{n^{1/3}}+(\frac{\sqrt{d}}{n\epsilon})^{1/2})$的精度。这一改进后的边界与当前寻找一阶稳定点的最优结果相匹配，表明差分隐私下寻找二阶稳定点可能无需付出额外计算代价。