Edge devices have typically been used for DNN inferencing. The increase in the compute power of accelerated edges is leading to their use in DNN training also. As privacy becomes a concern on multi-tenant edge devices, Docker containers provide a lightweight virtualization mechanism to sandbox models. But their overheads for edge devices are not yet explored. In this work, we study the impact of containerized DNN inference and training workloads on an NVIDIA AGX Orin edge device and contrast it against bare metal execution on running time, CPU, GPU and memory utilization, and energy consumption. Our analysis provides several interesting insights on these overheads.

翻译：边缘设备通常用于深度神经网络推理。随着加速边缘设备计算能力的提升，它们也被用于深度神经网络训练。当多租户边缘设备上的隐私问题日益突出时，Docker容器作为一种轻量级虚拟化机制可对模型进行沙盒隔离。然而，容器在边缘设备上的额外开销尚未得到充分研究。本研究以NVIDIA AGX Orin边缘设备为例，系统对比了容器化DNN推理与训练工作负载与裸机执行在运行时间、CPU、GPU及内存利用率、能耗等方面的差异。我们的分析揭示了关于这些额外开销的多项有趣发现。