Logistic regression training over encrypted data has been an attractive idea to security concerns for years. In this paper, we propose a faster gradient variant called $\texttt{quadratic gradient}$ for privacy-preserving logistic regression training. The core of $\texttt{quadratic gradient}$ can be seen as an extension of the simplified fixed Hessian. We enhance Nesterov's accelerated gradient (NAG) and Adaptive Gradient Algorithm (Adagrad) respectively with $\texttt{quadratic gradient}$ and evaluate the enhanced algorithms on several datasets. %gradient $ascent$ methods with this gradient variant on the gene dataset provided by the 2017 iDASH competition and other datasets. Experiments show that the enhanced methods have a state-of-the-art performance in convergence speed compared to the raw first-order gradient methods. We then adopt the enhanced NAG method to implement homomorphic logistic regression training, obtaining a comparable result by only $3$ iterations. There is a promising chance that $\texttt{quadratic gradient}$ could be used to enhance other first-order gradient methods for general numerical optimization problems.

翻译：多年来，对加密数据进行逻辑回归训练一直是应对安全问题的有吸引力的想法。本文提出一种名为$\texttt{quadratic gradient}$的更快梯度变体，用于隐私保护逻辑回归训练。$\texttt{quadratic gradient}$的核心可视为简化固定黑塞矩阵的扩展。我们分别使用$\texttt{quadratic gradient}$增强了Nesterov加速梯度（NAG）和自适应梯度算法（Adagrad），并在多个数据集上评估了增强后的算法。实验表明，与原始一阶梯度方法相比，增强方法在收敛速度上达到了当前最优性能。随后，我们采用增强后的NAG方法实现同态逻辑回归训练，仅需3次迭代即可获得可比结果。$\texttt{quadratic gradient}$有望用于增强通用数值优化问题中的其他一阶梯度方法。