The concept of attention, numerical weights that emphasize the importance of particular data, has proven to be very relevant in artificial intelligence. Relative entropy (RE, aka Kullback-Leibler divergence) plays a central role in communication theory. Here we combine these concepts, attention and RE. RE guides optimal encoding of messages in bandwidth-limited communication as well as optimal message decoding via the maximum entropy principle (MEP). In the coding scenario, RE can be derived from four requirements, namely being analytical, local, proper, and calibrated. Weighted RE, used for attention steering in communications, turns out to be improper. To see how proper attention communication can emerge, we analyze a scenario of a message sender who wants to ensure that the receiver of the message can perform well-informed actions. If the receiver decodes the message using the MEP, the sender only needs to know the receiver's utility function to inform optimally, but not the receiver's initial knowledge state. In case only the curvature of the utility function maxima are known, it becomes desirable to accurately communicate an attention function, in this case a by this curvature weighted and re-normalized probability function. Entropic attention communication is here proposed as the desired generalization of entropic communication that permits weighting while being proper, thereby aiding the design of optimal communication protocols in technical applications and helping to understand human communication. For example, our analysis shows how to derive the level of cooperation expected under misaligned interests of otherwise honest communication partners.

翻译：注意力概念，即强调特定数据重要性的数值权重，已在人工智能领域显示出高度相关性。相对熵（RE，亦称Kullback-Leibler散度）在通信理论中扮演核心角色。本文融合了注意力与相对熵这两个概念。相对熵指导带宽受限通信中的最优消息编码，以及通过最大熵原理（MEP）实现的最优消息解码。在编码场景下，相对熵可由四个要求推导得出：解析性、局部性、恰当性和校准性。用于通信中注意力引导的加权相对熵被证实是不恰当的。为探究恰当注意力通信的产生机制，我们分析了一个消息发送者希望确保接收者能基于充分信息采取行动的场景。若接收者使用最大熵原理解码消息，发送者只需知晓接收者的效用函数即可实现最优信息传递，而无需了解接收者的初始知识状态。当仅知效用函数最大值的曲率时，精确传递一个注意力函数便成为理想目标——该函数在此情况下为经此曲率加权并重新归一化的概率函数。本文提出熵注意力通信作为熵通信的预期推广形式，在保持恰当性的同时允许权重分配，从而助力技术应用中最优通信协议的设计，并促进对人类通信的理解。例如，我们的分析展示了如何在诚实通信伙伴利益不一致的情况下推导预期合作程度。