Transfer learning is a de facto standard method for efficiently training machine learning models for data-scarce problems by adding and fine-tuning new classification layers to a model pre-trained on large datasets. Although numerous previous studies proposed to use homomorphic encryption to resolve the data privacy issue in transfer learning in the machine learning as a service setting, most of them only focused on encrypted inference. In this study, we present HETAL, an efficient Homomorphic Encryption based Transfer Learning algorithm, that protects the client's privacy in training tasks by encrypting the client data using the CKKS homomorphic encryption scheme. HETAL is the first practical scheme that strictly provides encrypted training, adopting validation-based early stopping and achieving the accuracy of nonencrypted training. We propose an efficient encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods, and a highly precise softmax approximation algorithm with increased coverage. The experimental results for five well-known benchmark datasets show total training times of 567-3442 seconds, which is less than an hour.

翻译：迁移学习是通过在大数据集上预训练的模型上添加并微调新分类层，高效训练数据稀缺问题机器学习模型的事实标准方法。尽管先前众多研究提出在机器学习即服务场景下使用同态加密解决迁移学习中的数据隐私问题，但大多数仅关注加密推理。本研究提出HETAL，一种基于CKKS同态加密方案的高效同态加密迁移学习算法，通过在训练任务中加密客户端数据保护其隐私。HETAL是首个严格实现加密训练、采用基于验证的早停机制并达到非加密训练准确率的实用方案。我们提出一种高效加密矩阵乘法算法，其速度比先前方法快1.8至323倍，并设计了一种高精度、高覆盖率的softmax近似算法。在五个知名基准数据集上的实验结果显示，总训练时间为567-3442秒，即不到一小时。