Randomized sketching is a central tool for compressing large-scale optimization problems while preserving accuracy. In particular, sketches that are based on structured matrices, such as the Hadamard matrix, can be applied efficiently and often yield solutions that approximate those of the original problem at much lower computational cost. In differential privacy (DP), Gaussian sketching has been used to solve DP linear regression, beginning with \citet{sheffet2017differentially, sheffet2019old} and later refined by \citet{lev2025gaussianmix, lev2026near}. However, although these methods achieve strong utility guarantees, they usually do not improve runtime over classical DP approaches. In this work, we introduce a new DP sketching mechanism based on fast transforms, which, in certain cases, matches the runtime of classical fast sketching methods. We prove state-of-the-art privacy guarantees for this mechanism and show that, in favorable regimes, they match those of the Gaussian sketch up to a constant factor. As an application, we combine this mechanism with recent sketch-based methods for DP linear regression to obtain a new algorithm with strong utility and improved runtime. We establish privacy and accuracy guarantees for this algorithm, yielding, to the best of our knowledge, the first fast method for DP ordinary least squares.

翻译：随机化草图是压缩大规模优化问题同时保持精度的核心工具。特别地，基于结构化矩阵（如哈达玛矩阵）的草图可高效应用，且通常能以更低计算成本获得与原问题近似的结果。在差分隐私（DP）领域，高斯草图已被用于求解DP线性回归，该方法始于\citet{sheffet2017differentially, sheffet2019old}，后经\citet{lev2025gaussianmix, lev2026near}改进。然而，尽管这些方法具有强效用保证，其运行时间通常并未优于经典DP方法。本文提出一种基于快速变换的新型DP草图机制，该机制在特定情况下可与经典快速草图方法的运行时间持平。我们证明了该机制具有最先进的隐私保证，并在有利条件下表明其隐私保证与高斯草图相差不超过常数因子。作为应用，我们将该机制与近期基于草图的DP线性回归方法相结合，提出一种具有强效用和更优运行时间的新算法。我们为该算法建立了隐私与精度保证，据我们所知，这是首个用于DP普通最小二乘的快速方法。