The first large-scale deployment of private federated learning uses differentially private counting in the continual release model as a subroutine (Google AI blog titled "Federated Learning with Formal Differential Privacy Guarantees"). In this case, a concrete bound on the error is very relevant to reduce the privacy parameter. The standard mechanism for continual counting is the binary mechanism. We present a novel mechanism and show that its mean squared error is both asymptotically optimal and a factor 10 smaller than the error of the binary mechanism. We also show that the constants in our analysis are almost tight by giving non-asymptotic lower and upper bounds that differ only in the constants of lower-order terms. Our algorithm is a matrix mechanism for the counting matrix and takes constant time per release. We also use our explicit factorization of the counting matrix to give an upper bound on the excess risk of the private learning algorithm of Denisov et al. (NeurIPS 2022). Our lower bound for any continual counting mechanism is the first tight lower bound on continual counting under approximate differential privacy. It is achieved using a new lower bound on a certain factorization norm, denoted by $\gamma_F(\cdot)$, in terms of the singular values of the matrix. In particular, we show that for any complex matrix, $A \in \mathbb{C}^{m \times n}$, \[ \gamma_F(A) \geq \frac{1}{\sqrt{m}}\|A\|_1, \] where $\|\cdot \|$ denotes the Schatten-1 norm. We believe this technique will be useful in proving lower bounds for a larger class of linear queries. To illustrate the power of this technique, we show the first lower bound on the mean squared error for answering parity queries.

翻译：私人联邦学习的首次大规模部署使用持续发布模型下的差分隐私计数作为子程序（Google AI博客文章题为“具有正式差分隐私保证的联邦学习”）。在此情况下，误差的具体界对降低隐私参数至关重要。持续计数的标准机制是二进制机制。我们提出一种新颖机制，并证明其均方误差在渐近意义上最优，且比二进制机制的误差小一个数量级（10倍）。我们还通过给出非渐近的下界和上界（仅低阶项常数不同），表明分析中的常数近乎紧。我们的算法是一种用于计数矩阵的矩阵机制，每次发布耗时常数时间。我们还利用计数矩阵的显式分解，对Denisov等人（NeurIPS 2022）的私人学习算法给出了超额风险的上界。针对任何持续计数机制的下界，是近似差分隐私下持续计数的首个紧下界。该下界通过利用一种新的关于特定分解范数$\gamma_F(\cdot)$的下界实现，该下界以矩阵的奇异值表示。特别地，我们证明对任意复矩阵$A \in \mathbb{C}^{m \times n}$，有\[ \gamma_F(A) \geq \frac{1}{\sqrt{m}}\|A\|_1, \]其中$\|\cdot \|$表示Schatten-1范数。我们相信此技术将有助于证明更大类线性查询的下界。为了展示该技术的威力，我们首次给出了回答奇偶校验查询时均方误差的下界。