Distributed deep neural networks (DNNs) have emerged as a key technique to reduce communication overhead without sacrificing performance in edge computing systems. Recently, entropy coding has been introduced to further reduce the communication overhead. The key idea is to train the distributed DNN jointly with an entropy model, which is used as side information during inference time to adaptively encode latent representations into bit streams with variable length. To the best of our knowledge, the resilience of entropy models is yet to be investigated. As such, in this paper we formulate and investigate the resilience of entropy models to intentional interference (e.g., adversarial attacks) and unintentional interference (e.g., weather changes and motion blur). Through an extensive experimental campaign with 3 different DNN architectures, 2 entropy models and 4 rate-distortion trade-off factors, we demonstrate that the entropy attacks can increase the communication overhead by up to 95%. By separating compression features in frequency and spatial domain, we propose a new defense mechanism that can reduce the transmission overhead of the attacked input by about 9% compared to unperturbed data, with only about 2% accuracy loss. Importantly, the proposed defense mechanism is a standalone approach which can be applied in conjunction with approaches such as adversarial training to further improve robustness. Code will be shared for reproducibility.

翻译：分布式深度神经网络（DNN）已成为边缘计算系统中降低通信开销且不牺牲性能的关键技术。近期，熵编码被引入以进一步降低通信开销。其核心思想是将分布式DNN与熵模型联合训练，该模型在推理阶段作为辅助信息，将潜在表示自适应地编码为变长比特流。据我们所知，熵模型的鲁棒性尚未得到研究。为此，本文对熵模型在有意干扰（如对抗攻击）与无意干扰（如天气变化和运动模糊）下的鲁棒性进行建模与探究。通过涵盖3种不同DNN架构、2种熵模型及4种率失真权衡因子的广泛实验，我们证明熵攻击可使通信开销增加高达95%。通过将压缩特征在频域和空域进行分离，我们提出了一种新型防御机制，该机制相较于未受扰动数据可将受攻击输入的传输开销降低约9%，且准确率损失仅为2%。重要的是，所提出的防御机制是一种独立方法，可结合对抗训练等方案进一步改善鲁棒性。代码将开源以保障可复现性。