The success of large-scale language models has established tokens as compact and meaningful units for natural-language representation, which motivates token communication over wireless channels, where tokens are considered fundamental units for wireless transmission. We propose a context-aware token communication framework that uses a pretrained masked language model (MLM) as a shared contextual probability model between the transmitter (Tx) and receiver (Rx). At Rx, we develop an iterative token detection method that jointly exploits MLM-guided contextual priors and channel observations based on a Bayesian perspective. At Tx, we additionally introduce a context-aware masking strategy which skips highly predictable token transmission to reduce transmission rate. Simulation results demonstrate that the proposed framework substantially improves reconstructed sentence quality and supports effective rate adaptation under various channel conditions.

翻译：大规模语言模型的成功使得令牌成为自然语言表示的紧凑且有意义的基本单元，这启发了在无线信道中进行令牌通信的构想，即将令牌视为无线传输的基本单元。我们提出了一种上下文感知的令牌通信框架，该框架使用一个预训练的掩码语言模型（MLM）作为发射端（Tx）与接收端（Rx）之间共享的上下文概率模型。在接收端，我们基于贝叶斯视角，开发了一种迭代令牌检测方法，该方法联合利用了MLM引导的上下文先验知识与信道观测信息。在发射端，我们额外引入了一种上下文感知的掩码策略，通过跳过传输高度可预测的令牌来降低传输速率。仿真结果表明，所提框架显著提升了重建句子的质量，并在各种信道条件下支持有效的速率自适应。