Currently, there is a burgeoning demand for deploying deep learning (DL) models on ubiquitous edge Internet of Things (IoT) devices attributed to their low latency and high privacy preservation. However, DL models are often large in size and require large-scale computation, which prevents them from being placed directly onto IoT devices, where resources are constrained and 32-bit floating-point (float-32) operations are unavailable. Commercial framework (i.e., a set of toolkits) empowered model quantization is a pragmatic solution that enables DL deployment on mobile devices and embedded systems by effortlessly post-quantizing a large high-precision model (e.g., float-32) into a small low-precision model (e.g., int-8) while retaining the model inference accuracy. However, their usability might be threatened by security vulnerabilities. This work reveals that the standard quantization toolkits can be abused to activate a backdoor. We demonstrate that a full-precision backdoored model which does not have any backdoor effect in the presence of a trigger -- as the backdoor is dormant -- can be activated by the default i) TensorFlow-Lite (TFLite) quantization, the only product-ready quantization framework to date, and ii) the beta released PyTorch Mobile framework. When each of the float-32 models is converted into an int-8 format model through the standard TFLite or Pytorch Mobile framework's post-training quantization, the backdoor is activated in the quantized model, which shows a stable attack success rate close to 100% upon inputs with the trigger, while it behaves normally upon non-trigger inputs. This work highlights that a stealthy security threat occurs when an end user utilizes the on-device post-training model quantization frameworks, informing security researchers of cross-platform overhaul of DL models post quantization even if these models pass front-end backdoor inspections.

翻译：当前，将深度学习模型部署到资源受限的边缘物联网设备上的需求日益增长，这得益于其低延迟和高隐私保护特性。然而，深度学习模型通常规模庞大且需要大规模计算能力，这使其无法直接部署到仅支持定点运算且缺乏32位浮点（float-32）操作能力的物联网设备上。基于商业框架（即工具集）的模型量化是一种实用解决方案，通过将大型高精度模型（如float-32）便捷地后量化为小型低精度模型（如int-8），在保持模型推理精度的同时实现移动端与嵌入式系统的部署。然而，这些框架的可用性可能受到安全漏洞的威胁。本工作揭示了标准量化工具包可能被滥用激活后门。我们证明，一个全精度后门模型在触发条件下本无任何后门效应（即后门处于休眠状态），但可通过以下两种方式被激活：i）当前唯一可产品化的量化框架TensorFlow-Lite（TFLite）的默认量化机制，以及ii）测试版发布的PyTorch Mobile框架。当这些float-32模型通过标准TFLite或PyTorch Mobile框架的后训练量化转换为int-8格式模型时，后门在量化模型中被激活。该量化模型对含触发器的输入保持接近100%的稳定攻击成功率，而对不含触发器的输入则表现正常。本工作揭示了终端用户利用设备端后训练模型量化框架时存在的隐蔽安全威胁，提醒安全研究者即使模型通过前端后门检测，仍需对其后量化结果进行跨平台彻底审查。