We study the problem of approximating stationary points of Lipschitz and smooth functions under $(\varepsilon,\delta)$-differential privacy (DP) in both the finite-sum and stochastic settings. A point $\widehat{w}$ is called an $\alpha$-stationary point of a function $F:\mathbb{R}^d\rightarrow\mathbb{R}$ if $\|\nabla F(\widehat{w})\|\leq \alpha$. We provide a new efficient algorithm that finds an $\tilde{O}\big(\big[\frac{\sqrt{d}}{n\varepsilon}\big]^{2/3}\big)$-stationary point in the finite-sum setting, where $n$ is the number of samples. This improves on the previous best rate of $\tilde{O}\big(\big[\frac{\sqrt{d}}{n\varepsilon}\big]^{1/2}\big)$. We also give a new construction that improves over the existing rates in the stochastic optimization setting, where the goal is to find approximate stationary points of the population risk. Our construction finds a $\tilde{O}\big(\frac{1}{n^{1/3}} + \big[\frac{\sqrt{d}}{n\varepsilon}\big]^{1/2}\big)$-stationary point of the population risk in time linear in $n$. Furthermore, under the additional assumption of convexity, we completely characterize the sample complexity of finding stationary points of the population risk (up to polylog factors) and show that the optimal rate on population stationarity is $\tilde \Theta\big(\frac{1}{\sqrt{n}}+\frac{\sqrt{d}}{n\varepsilon}\big)$. Finally, we show that our methods can be used to provide dimension-independent rates of $O\big(\frac{1}{\sqrt{n}}+\min\big(\big[\frac{\sqrt{rank}}{n\varepsilon}\big]^{2/3},\frac{1}{(n\varepsilon)^{2/5}}\big)\big)$ on population stationarity for Generalized Linear Models (GLM), where $rank$ is the rank of the design matrix, which improves upon the previous best known rate.

翻译：我们研究在 $( varepsilon,\ delta) 和 dispreaty (DP) 的固定点 。 $( ww) 被称作 $( a) 的固定点 。 如果 $( a) f( a) f( a) f( a) f( a) f( b) f( b) f( b) f( b) f( b) f( b) dleq( d) leqqq( t) 美元 。 我们提供一种新的高效算法, 找到 $( t) 和 d( t) di( di) a( di) 。 美元1\\\\\\\ di) a( t) a( di) a( lad) ladq( q) 。 美元是样本数量。 这比以前最好的速率( \\\\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \\\\\\\\\\ di) a( di) lab) lab) a( lax) lax) lax) a( lax) mo( lax) a( lax) a) lax) lax) lax) mod) la mod) a( lax) a( lax) lax) mod) lax) lax) a laus ( mo( lax) mod) lax) lax( lax) lax ( lax( mo( lax) lax) lax( lax) lax) a( lax) lax) lax) mo( lax) lax) lax) lax) lax ( lax) la la mo( lax) mo( mo( lax) a) a( lax) a( mo) a( lax) a( mo) lax) a( lax) la