The rise of IoT devices has prompted the demand for deploying machine learning at-the-edge with real-time, efficient, and secure data processing. In this context, implementing machine learning (ML) models with real-valued weight parameters can prove to be impractical particularly for large models, and there is a need to train models with quantized discrete weights. At the same time, these low-dimensional models also need to preserve privacy of the underlying dataset. In this work, we present RQP-SGD, a new approach for privacy-preserving quantization to train machine learning models for low-memory ML-at-the-edge. This approach combines differentially private stochastic gradient descent (DP-SGD) with randomized quantization, providing a measurable privacy guarantee in machine learning. In particular, we study the utility convergence of implementing RQP-SGD on ML tasks with convex objectives and quantization constraints and demonstrate its efficacy over deterministic quantization. Through experiments conducted on two datasets, we show the practical effectiveness of RQP-SGD.

翻译：物联网设备的普及催生了在边缘端部署机器学习的需求，要求实现实时、高效且安全的数据处理。在此背景下，使用实值权重参数的机器学习模型尤其对大型模型而言可能不切实际，因此需要训练具有量化离散权重的模型。同时，这些低维模型还需保护底层数据集的隐私。本文提出了一种新的隐私保护量化方法RQP-SGD，用于训练面向低内存边缘端机器学习的模型。该方法将差分隐私随机梯度下降与随机量化相结合，提供了机器学习中可量化的隐私保障。我们重点研究了在具有凸目标函数与量化约束的机器学习任务中，RQP-SGD的效用收敛性，并证明了其相对于确定性量化的优势。通过在两个数据集上的实验，我们展示了RQP-SGD的实际有效性。