Trusted Execution Environments (TEE) are used to safeguard on-device models. However, directly employing TEEs to secure the entire DNN model is challenging due to the limited computational speed. Utilizing GPU can accelerate DNN's computation speed but commercial widely-available GPUs usually lack security protection. To this end, scholars introduce TSDP, a method that protects privacy-sensitive weights within TEEs and offloads insensitive weights to GPUs. Nevertheless, current methods do not consider the presence of a knowledgeable adversary who can access abundant publicly available pre-trained models and datasets. This paper investigates the security of existing methods against such a knowledgeable adversary and reveals their inability to fulfill their security promises. Consequently, we introduce a novel partition before training strategy, which effectively separates privacy-sensitive weights from other components of the model. Our evaluation demonstrates that our approach can offer full model protection with a computational cost reduced by a factor of 10. In addition to traditional CNN models, we also demonstrate the scalability to large language models. Our approach can compress the private functionalities of the large language model to lightweight slices and achieve the same level of protection as the shielding-whole-model baseline.

翻译：可信执行环境（TEE）被用于保护设备端模型。然而，由于计算速度有限，直接使用TEE来保护整个深度神经网络模型具有挑战性。利用GPU可以加速DNN的计算速度，但商业上广泛可用的GPU通常缺乏安全保护。为此，学者们提出了TSDP方法，该方法将隐私敏感的权重保护在TEE内，并将不敏感的权重卸载到GPU上。然而，现有方法并未考虑存在一个知识渊博的对手，该对手能够访问大量公开可用的预训练模型和数据集。本文研究了现有方法在面对此类知识渊博对手时的安全性，并揭示了它们无法兑现其安全承诺。因此，我们引入了一种新颖的训练前划分策略，该策略能有效将隐私敏感权重与模型的其他组件分离。我们的评估表明，我们的方法能以计算成本降低10倍的代价提供完整的模型保护。除了传统的CNN模型外，我们还展示了该方法对大型语言模型的可扩展性。我们的方法能够将大型语言模型的私有功能压缩为轻量级切片，并实现与全模型屏蔽基线相同的保护水平。