The formal privacy guarantee provided by Differential Privacy (DP) bounds the leakage of sensitive information from deep learning models. In practice, however, this comes at a severe computation and accuracy cost. The recently established state of the art (SOTA) results in image classification under DP are due to the use of heavy data augmentation and large batch sizes, leading to a drastically increased computation overhead. In this work, we propose to use more efficient models with improved feature quality by introducing steerable equivariant convolutional networks for DP training. We demonstrate that our models are able to outperform the current SOTA performance on CIFAR-10 by up to $9\%$ across different $\varepsilon$-values while reducing the number of model parameters by a factor of $35$ and decreasing the computation time by more than $90 \%$. Our results are a large step towards efficient model architectures that make optimal use of their parameters and bridge the privacy-utility gap between private and non-private deep learning for computer vision.

翻译：差分隐私（DP）提供的正式隐私保证限制了对深度学习模型敏感信息的泄露。然而在实践中，这带来了严重的计算和准确率代价。最近在差分隐私条件下的图像分类领域，现有最优结果得益于大量数据增强和大批量（batch size）的使用，导致计算开销急剧增加。本研究通过引入可操控等变卷积网络用于差分隐私训练，提出使用特征质量更优的高效模型。我们证明，在不同$\varepsilon$值条件下，模型在CIFAR-10数据集上的性能较当前最优结果高出最多$9\%$，同时模型参数数量减少$35$倍，计算时间降低超过$90\%$。我们的成果朝着高效模型架构迈出了重要一步，这类架构能充分利用参数，弥合计算机视觉领域私有与非私有深度学习之间的隐私-效用差距。