We initiate a systematic study of worst-group risk minimization under $(\epsilon, \delta)$-differential privacy (DP). The goal is to privately find a model that approximately minimizes the maximal risk across $p$ sub-populations (groups) with different distributions, where each group distribution is accessed via a sample oracle. We first present a new algorithm that achieves excess worst-group population risk of $\tilde{O}(\frac{p\sqrt{d}}{K\epsilon} + \sqrt{\frac{p}{K}})$, where $K$ is the total number of samples drawn from all groups and $d$ is the problem dimension. Our rate is nearly optimal when each distribution is observed via a fixed-size dataset of size $K/p$. Our result is based on a new stability-based analysis for the generalization error. In particular, we show that $\Delta$-uniform argument stability implies $\tilde{O}(\Delta + \frac{1}{\sqrt{n}})$ generalization error w.r.t. the worst-group risk, where $n$ is the number of samples drawn from each sample oracle. Next, we propose an algorithmic framework for worst-group population risk minimization using any DP online convex optimization algorithm as a subroutine. Hence, we give another excess risk bound of $\tilde{O}\left( \sqrt{\frac{d^{1/2}}{\epsilon K}} +\sqrt{\frac{p}{K\epsilon^2}} \right)$. Assuming the typical setting of $\epsilon=\Theta(1)$, this bound is more favorable than our first bound in a certain range of $p$ as a function of $K$ and $d$. Finally, we study differentially private worst-group empirical risk minimization in the offline setting, where each group distribution is observed by a fixed-size dataset. We present a new algorithm with nearly optimal excess risk of $\tilde{O}(\frac{p\sqrt{d}}{K\epsilon})$.

翻译：我们系统性地研究了在 $(\epsilon, \delta)$-差分隐私（DP）约束下的最差组风险最小化问题。目标是私密地找到一个模型，使其近似最小化 $p$ 个不同分布的子总体（组）上的最大风险，其中每个组分布通过样本预言机访问。我们首先提出一种新算法，实现了 $\tilde{O}(\frac{p\sqrt{d}}{K\epsilon} + \sqrt{\frac{p}{K}})$ 的超额最差组总体风险，其中 $K$ 是从所有组抽取的总样本数，$d$ 是问题维度。当每个分布通过大小为 $K/p$ 的固定数据集观测时，我们的速率几乎最优。该结果基于一种新的基于稳定性的泛化误差分析。特别地，我们证明 $\Delta$-一致参数稳定性意味着关于最差组风险的 $\tilde{O}(\Delta + \frac{1}{\sqrt{n}})$ 泛化误差，其中 $n$ 是从每个样本预言机抽取的样本数。接着，我们提出一种利用任意 DP 在线凸优化算法作为子程序的最差组总体风险最小化算法框架。由此得到另一个超额风险界 $\tilde{O}\left( \sqrt{\frac{d^{1/2}}{\epsilon K}} +\sqrt{\frac{p}{K\epsilon^2}} \right)$。假设 $\epsilon=\Theta(1)$ 的典型设定，该界在 $p$ 关于 $K$ 和 $d$ 的某些范围内优于我们的第一个界。最后，我们研究离线设置下的差分隐私最差组经验风险最小化问题，其中每个组分布通过固定大小的数据集观测。我们提出一种新算法，实现了近乎最优的超额风险 $\tilde{O}(\frac{p\sqrt{d}}{K\epsilon})$。