The recent advancements in deep learning have brought about significant changes in various aspects of people's lives. Meanwhile, these rapid developments have raised concerns about the legitimacy of the training process of deep networks. However, to protect the intellectual properties of untrusted AI developers, directly examining the training process by accessing the model parameters and training data by verifiers is often prohibited. In response to this challenge, we present zkDL, an efficient zero-knowledge proof of deep learning training. At the core of zkDL is zkReLU, a specialized zero-knowledge proof protocol with optimized proving time and proof size for the ReLU activation function, a major obstacle in verifiable training due to its non-arithmetic nature. To integrate zkReLU into the proof system for the entire training process, we devise a novel construction of an arithmetic circuit from neural networks. By leveraging the abundant parallel computation resources, this construction reduces proving time and proof sizes by a factor of the network depth. As a result, zkDL enables the generation of complete and sound proofs, taking less than a minute with a size of less than 20 kB per training step, for a 16-layer neural network with 200M parameters, while ensuring the privacy of data and model parameters.

翻译：近期深度学习的进展给人们生活的方方面面带来了显著变化。与此同时，这些快速发展也引发了人们对深度网络训练过程合法性的担忧。然而，为了保护不可信AI开发者的知识产权，验证者通常无法直接访问模型参数和训练数据来检查训练过程。针对这一挑战，我们提出了zkDL，一种高效的深度学习训练零知识证明方法。zkDL的核心是zkReLU，一种针对ReLU激活函数优化的专用零知识证明协议，具有优化的证明时间和证明大小，而ReLU因其非算术特性一直是可验证训练中的主要障碍。为了将zkReLU整合到整个训练过程的证明系统中，我们设计了一种新颖的从神经网络构建算术电路的方法。通过利用丰富的并行计算资源，这种构建方法将证明时间和证明大小降低了网络深度倍。结果，zkDL能够为具有2亿参数的16层神经网络生成完整且可靠的证明，每个训练步骤耗时不到一分钟，大小小于20 kB，同时确保数据和模型参数的隐私性。