Collaborative inference systems are one of the emerging solutions for deploying deep neural networks (DNNs) at the wireless network edge. Their main idea is to divide a DNN into two parts, where the first is shallow enough to be reliably executed at edge devices of limited computational power, while the second part is executed at an edge server with higher computational capabilities. The main advantage of such systems is that the input of the DNN gets compressed as the subsequent layers of the shallow part extract only the information necessary for the task. As a result, significant communication savings can be achieved compared to transmitting raw input samples. In this work, we study early exiting in the context of collaborative inference, which allows obtaining inference results at the edge device for certain samples, without the need to transmit the partially processed data to the edge server at all, leading to further communication savings. The central part of our system is the transmission-decision (TD) mechanism, which, given the information from the early exit, and the wireless channel conditions, decides whether to keep the early exit prediction or transmit the data to the edge server for further processing. In this paper, we evaluate various TD mechanisms and show experimentally, that for an image classification task over the wireless edge, proper utilization of early exits can provide both performance gains and significant communication savings.

翻译：协同推理系统是部署深度神经网络（DNN）于无线网络边缘的新兴解决方案之一。其核心思想是将DNN划分为两部分：第一部分足够浅层，可在计算能力有限的边缘设备上可靠执行；第二部分则在算力更强的边缘服务器上运行。此类系统的主要优势在于，DNN的输入会在浅层部分的后续层中自动压缩，仅提取任务所需的关键信息。因此，相较于直接传输原始输入样本，可显著节省通信开销。本研究探讨了早期退出机制在协同推理中的应用——该机制允许在边缘设备上直接获取部分样本的推理结果，完全无需将预处理数据传输至边缘服务器，从而进一步降低通信成本。系统的核心是传输决策（TD）机制，该机制根据早期退出阶段的信息与无线信道状态，决定保留早期退出预测结果还是将数据传输至边缘服务器进行后续处理。本文评估了多种TD机制，并通过实验证明：在无线边缘的图像分类任务中，合理利用早期退出既能提升性能，又可实现显著的通信成本节约。