We present a sample- and time-efficient differentially private algorithm for ordinary least squares, with error that depends linearly on the dimension and is independent of the condition number of $X^\top X$, where $X$ is the design matrix. All prior private algorithms for this task require either $d^{3/2}$ examples, error growing polynomially with the condition number, or exponential time. Our near-optimal accuracy guarantee holds for any dataset with bounded statistical leverage and bounded residuals. Technically, we build on the approach of Brown et al. (2023) for private mean estimation, adding scaled noise to a carefully designed stable nonprivate estimator of the empirical regression vector.

翻译：我们提出了一种样本和时间高效的差分隐私算法，用于普通最小二乘法，其误差与维度呈线性关系，且与设计矩阵$X$的$X^\top X$条件数无关。所有先前的私有算法要么需要$d^{3/2}$个样本，误差随条件数多项式增长，要么需要指数时间。我们的近最优精度保证对任何具有有界统计杠杆和有界残差的数据集都成立。技术上，我们基于Brown等人（2023）的私有均值估计方法，通过对经验回归向量精心设计的稳定非私有估计量添加缩放噪声来实现。