Collaborative inference has been a promising solution to enable resource-constrained edge devices to perform inference using state-of-the-art deep neural networks (DNNs). In collaborative inference, the edge device first feeds the input to a partial DNN locally and then uploads the intermediate result to the cloud to complete the inference. However, recent research indicates model inversion attacks (MIAs) can reconstruct input data from intermediate results, posing serious privacy concerns for collaborative inference. Existing perturbation and cryptography techniques are inefficient and unreliable in defending against MIAs while performing accurate inference. This paper provides a viable solution, named PATROL, which develops privacy-oriented pruning to balance privacy, efficiency, and utility of collaborative inference. PATROL takes advantage of the fact that later layers in a DNN can extract more task-specific features. Given limited local resources for collaborative inference, PATROL intends to deploy more layers at the edge based on pruning techniques to enforce task-specific features for inference and reduce task-irrelevant but sensitive features for privacy preservation. To achieve privacy-oriented pruning, PATROL introduces two key components: Lipschitz regularization and adversarial reconstruction training, which increase the reconstruction errors by reducing the stability of MIAs and enhance the target inference model by adversarial training, respectively.

翻译：协作推理已成为一种有前景的解决方案，使资源受限的边缘设备能够利用最先进的深度神经网络（DNN）进行推理。在协作推理中，边缘设备首先将输入馈送至本地部分DNN，然后将中间结果上传至云端以完成推理。然而，近期研究表明，模型反转攻击（MIA）能从中间结果重建输入数据，为协作推理带来严重的隐私问题。现有扰动和密码学技术在防御MIA时效率低下且不可靠，同时难以保证精确推理。本文提出一种名为PATROL的可行方案，其开发了隐私导向剪枝方法，以平衡协作推理的隐私性、效率和效用性。PATROL利用DNN中后层能提取更多任务特定特征的事实。在协作推理面临本地资源限制的情况下，PATROL旨在基于剪枝技术在边缘部署更多层，以强制提取推理所需的任务特定特征，同时减少与任务无关但敏感的隐私保护特征。为实现隐私导向剪枝，PATROL引入了两个关键组件：Lipschitz正则化与对抗性重建训练，前者通过降低MIA的稳定性来增加重建误差，后者通过对抗训练增强目标推理模型。