Logistic regression training over encrypted data has been an attractive idea to security concerns for years. In this paper, we propose a faster gradient variant called $\texttt{quadratic gradient}$ for privacy-preserving logistic regression training. The core of $\texttt{quadratic gradient}$ can be seen as an extension of the simplified fixed Hessian. We enhance Nesterov's accelerated gradient (NAG) and Adaptive Gradient Algorithm (Adagrad) respectively with $\texttt{quadratic gradient}$ and evaluate the enhanced algorithms on several datasets. %gradient $ascent$ methods with this gradient variant on the gene dataset provided by the 2017 iDASH competition and other datasets. Experiments show that the enhanced methods have a state-of-the-art performance in convergence speed compared to the raw first-order gradient methods. We then adopt the enhanced NAG method to implement homomorphic logistic regression training, obtaining a comparable result by only $3$ iterations. There is a promising chance that $\texttt{quadratic gradient}$ could be used to enhance other first-order gradient methods for general numerical optimization problems.

翻译：多年来，对加密数据进行逻辑回归训练一直是解决安全问题的吸引人思路。本文提出了一种名为$\texttt{quadratic gradient}$（二次梯度）的更快梯度变体，用于隐私保护逻辑回归训练。该二次梯度的核心可视为简化固定海森矩阵的扩展。我们分别将Nesterov加速梯度（NAG）和自适应梯度算法（Adagrad）与二次梯度相结合，并在多个数据集上评估了增强后的算法。实验表明，与原始一阶梯度方法相比，增强方法在收敛速度上达到了最先进的性能。随后，我们采用增强的NAG方法实现了同态逻辑回归训练，仅需3次迭代即可获得可比较的结果。二次梯度有望被用于增强一般数值优化问题中的其他一阶梯度方法。