Conventional retransmission (ARQ) protocols are designed with the goal of ensuring the correct reception of all the individual transmitter's packets at the receiver. When the transmitter is a learner communicating with a teacher, this goal is at odds with the actual aim of the learner, which is that of eliciting the most relevant label information from the teacher. Taking an active learning perspective, this paper addresses the following key protocol design questions: (i) Active batch selection: Which batch of inputs should be sent to the teacher to acquire the most useful information and thus reduce the number of required communication rounds? (ii) Batch encoding: Can batches of data points be combined to reduce the communication resources required at each communication round? Specifically, this work introduces Communication-Constrained Bayesian Active Knowledge Distillation (CC-BAKD), a novel protocol that integrates Bayesian active learning with compression via a linear mix-up mechanism. Comparisons with existing active learning protocols demonstrate the advantages of the proposed approach.

翻译：传统自动重传请求（ARQ）协议旨在确保接收端能够正确接收发送端的所有独立数据包。当发送端作为学习者与教师进行通信时，该目标与学习者的实际目标（即从教师处获取最相关的标签信息）存在矛盾。本文从主动学习的视角，探讨以下关键协议设计问题：（i）主动批次选择：应向教师发送哪些输入批次以获取最有用的信息，从而减少所需通信轮数？（ii）批次编码：能否将数据点批次组合以减少每轮通信所需的资源？具体而言，本文提出了一种新型协议——通信受限的贝叶斯主动知识蒸馏（CC-BAKD），该协议通过线性混合机制将贝叶斯主动学习与数据压缩相结合。与现有主动学习协议的对比验证了所提方法的优越性。