Cloud workloads have dominated generative AI based on large language models (LLM). Specialized hardware accelerators, such as GPUs, NPUs, and TPUs, play a key role in AI adoption due to their superior performance over general-purpose CPUs. The AI models and the data are often highly sensitive and come from mutually distrusting parties. Existing CPU-based TEEs such as Intel SGX or AMD SEV do not provide sufficient protection. Device-centric TEEs like Nvidia-CC only address tightly coupled CPU-GPU systems with a proprietary solution requiring TEE on the host CPU side. On the other hand, existing academic proposals are tailored toward specific CPU-TEE platforms. To address this gap, we propose Ascend-CC, a confidential computing architecture based on discrete NPU devices that requires no trust in the host system. Ascend-CC provides strong security by ensuring data and model encryption that protects not only the data but also the model parameters and operator binaries. Ascend-CC uses delegation-based memory semantics to ensure isolation from the host software stack, and task attestation provides strong model integrity guarantees. Our Ascend-CC implementation and evaluation with state-of-the-art LLMs such as Llama2 and Llama3 shows that Ascend-CC introduces minimal overhead with no changes in the AI software stack.

翻译：基于大语言模型（LLM）的生成式人工智能工作负载已在云端占据主导地位。GPU、NPU和TPU等专用硬件加速器因其性能远超通用CPU，在人工智能应用部署中发挥着关键作用。AI模型及其数据通常具有高度敏感性，且常来自互不信任的参与方。现有基于CPU的可信执行环境（TEE），如Intel SGX或AMD SEV，无法提供充分保护。Nvidia-CC等以设备为中心的TEE方案仅适用于紧耦合的CPU-GPU系统，且需依赖主机CPU端的TEE实现，属于专有解决方案。另一方面，现有学术提案多针对特定CPU-TEE平台定制。为填补这一空白，我们提出Ascend-CC——一种基于独立NPU设备的机密计算架构，无需信任主机系统。Ascend-CC通过确保数据和模型加密提供强安全保护，不仅保护数据，同时保护模型参数与算子二进制文件。该架构采用基于委托的内存语义确保与主机软件栈的隔离，并通过任务证明机制提供强模型完整性保证。我们基于Llama2、Llama3等前沿大语言模型对Ascend-CC的实现与评估表明，该架构在无需修改AI软件栈的前提下仅引入极低开销。