Deep neural network (DNN) models have become prevalent in edge devices for real-time inference. However, they are vulnerable to model extraction attacks and require protection. Existing defense approaches either fail to fully safeguard model confidentiality or result in significant latency issues. To overcome these challenges, this paper presents MirrorNet, which leverages Trusted Execution Environment (TEE) to enable secure on-device DNN inference. It generates a TEE-friendly implementation for any given DNN model to protect the model confidentiality, while meeting the stringent computation and storage constraints of TEE. The framework consists of two key components: the backbone model (BackboneNet), which is stored in the normal world but achieves lower inference accuracy, and the Companion Partial Monitor (CPM), a lightweight mirrored branch stored in the secure world, preserving model confidentiality. During inference, the CPM monitors the intermediate results from the BackboneNet and rectifies the classification output to achieve higher accuracy. To enhance flexibility, MirrorNet incorporates two modules: the CPM Strategy Generator, which generates various protection strategies, and the Performance Emulator, which estimates the performance of each strategy and selects the most optimal one. Extensive experiments demonstrate the effectiveness of MirrorNet in providing security guarantees while maintaining low computation latency, making MirrorNet a practical and promising solution for secure on-device DNN inference. For the evaluation, MirrorNet can achieve a 18.6% accuracy gap between authenticated and illegal use, while only introducing 0.99% hardware overhead.

翻译：深度神经网络（DNN）模型已广泛应用于边缘设备的实时推理场景。然而，此类模型易受模型窃取攻击，亟需保护措施。现有防御方法要么无法完全保障模型机密性，要么引发严重的延迟问题。为应对上述挑战，本文提出MirrorNet框架，利用可信执行环境（TEE）实现安全的设备端DNN推理。该框架可为任意给定DNN模型生成TEE友好的实现方案，在满足TEE严格计算与存储约束的同时，保护模型机密性。其核心包含两大组件：存储于普通世界的骨干网络（BackboneNet），其推理精度较低；以及安全世界中的轻量级镜像分支——伴生部分监控器（CPM），用于维护模型机密性。推理过程中，CPM监控BackboneNet的中间结果并修正分类输出，从而实现更高精度。为增强灵活性，MirrorNet集成两个模块：CPM策略生成器可产生多种保护策略，性能模拟器则评估各策略性能并择优选取。大量实验表明，MirrorNet在维持低计算延迟的同时，能够有效提供安全保障，成为安全设备端DNN推理的实用化解决方案。评测结果显示，MirrorNet可在认证使用与非法使用间实现18.6%的精度差距，且硬件开销仅为0.99%。