We study the problem of differentially-private (DP) stochastic (convex-concave) saddle-points in the polyhedral setting. We propose $(\varepsilon, \delta)$-DP algorithms based on stochastic mirror descent that attain nearly dimension-independent convergence rates for the expected duality gap, a type of guarantee that was known before only for bilinear objectives. For convex-concave and first-order-smooth stochastic objectives, our algorithms attain a rate of $\sqrt{\log(d)/n} + (\log(d)^{3/2}/[n\varepsilon])^{1/3}$, where $d$ is the dimension of the problem and $n$ the dataset size. Under an additional second-order-smoothness assumption, we improve the rate on the expected gap to $\sqrt{\log(d)/n} + (\log(d)^{3/2}/[n\varepsilon])^{2/5}$. Under this additional assumption, we also show, by using bias-reduced gradient estimators, that the duality gap is bounded by $\log(d)/\sqrt{n} + \log(d)/[n\varepsilon]^{1/2}$ with constant success probability. This result provides evidence of the near-optimality of the approach. Finally, we show that combining our methods with acceleration techniques from online learning leads to the first algorithm for DP Stochastic Convex Optimization in the polyhedral setting that is not based on Frank-Wolfe methods. For convex and first-order-smooth stochastic objectives, our algorithms attain an excess risk of $\sqrt{\log(d)/n} + \log(d)^{7/10}/[n\varepsilon]^{2/5}$, and when additionally assuming second-order-smoothness, we improve the rate to $\sqrt{\log(d)/n} + \log(d)/\sqrt{n\varepsilon}$. Instrumental to all of these results are various extensions of the classical Maurey Sparsification Lemma, which may be of independent interest.

翻译：我们研究了多面体设定下差分隐私（DP）随机（凸-凹）鞍点问题。我们提出了基于随机镜像下降的$(\varepsilon, \delta)$-DP算法，该算法在期望对偶间隙上实现了近乎维度无关的收敛速率——此类保证此前仅对双线性目标函数已知。针对凸-凹且一阶光滑的随机目标，我们的算法达到$\sqrt{\log(d)/n} + (\log(d)^{3/2}/[n\varepsilon])^{1/3}$的速率，其中$d$为问题维度，$n$为数据集规模。在附加二阶光滑性假设下，我们将期望间隙的速率改进至$\sqrt{\log(d)/n} + (\log(d)^{3/2}/[n\varepsilon])^{2/5}$。在此附加假设下，通过使用偏差缩减梯度估计量，我们进一步表明对偶间隙以常值成功概率受限于$\log(d)/\sqrt{n} + \log(d)/[n\varepsilon]^{1/2}$。该结果验证了方法的近乎最优性。最后，我们证明将所提方法与在线学习的加速技术相结合，可得到多面体设定下首个不基于Frank-Wolfe方法的DP随机凸优化算法。针对凸且一阶光滑的随机目标，我们的算法达到超额风险$\sqrt{\log(d)/n} + \log(d)^{7/10}/[n\varepsilon]^{2/5}$，而在附加二阶光滑性假设时，该速率可改进至$\sqrt{\log(d)/n} + \log(d)/\sqrt{n\varepsilon}$。所有这些结果的关键工具是经典Maurey稀疏化引理的各种推广，这些推广本身可能具有独立研究价值。