Training large-scale CNNs that during inference can be run under Homomorphic Encryption (HE) is challenging due to the need to use only polynomial operations. This limits HE-based solutions adoption. We address this challenge and pioneer in providing a novel training method for large polynomial CNNs such as ResNet-152 and ConvNeXt models, and achieve promising accuracy on encrypted samples on large-scale dataset such as ImageNet. Additionally, we provide optimization insights regarding activation functions and skip-connection latency impacts, enhancing HE-based evaluation efficiency. Finally, to demonstrate the robustness of our method, we provide a polynomial adaptation of the CLIP model for secure zero-shot prediction, unlocking unprecedented capabilities at the intersection of HE and transfer learning.

翻译：针对同态加密（HE）环境下推理时需仅使用多项式运算的约束，大规模卷积神经网络的训练极具挑战性，这限制了基于HE解决方案的推广。我们率先提出一种针对大规模多项式卷积网络（如ResNet-152和ConvNeXt模型）的新型训练方法，并在ImageNet等大规模数据集的加密样本上取得了具有竞争力的准确率。此外，我们针对激活函数与跳跃连接延迟影响提出优化见解，提升了基于HE的推理效率。最后，为验证方法的鲁棒性，我们提出CLIP模型的多项式适配方案，实现了安全的零样本预测，开创性地融合了同态加密与迁移学习的前沿能力。