Large language models, LLMs, are increasingly deployed in multiturn settings where earlier responses shape later ones, making reliability dependent on whether a conversation remains consistent over time. When this consistency degrades undetected, downstream decisions lose their grounding in the exchange that produced them. Yet current evaluation methods assess isolated outputs rather than the interaction producing them. Here we show that conversational structural consistency can be monitored directly from token frequency statistics, without embeddings, auxiliary evaluators or access to model internals. We formalize this signal as Bipredictability, P, which measures shared predictability across the context, response, next prompt loop relative to the turn total uncertainty, and implement it in a lightweight auxiliary architecture, the Information Digital Twin, IDT. Across 4,574 conversational turns spanning 34 conditions, one student model and three frontier teacher models, P established a stable runtime baseline, aligned with structural consistency in 85 percent of conditions but with semantic quality in only 44 percent, and the IDT detected all tested contradictions, topic shifts and non-sequiturs with 100 percent sensitivity. These results show that reliability in extended LLM interaction cannot be reduced to response quality alone, and that structural monitoring from the observable token stream can complement semantic evaluation in deployment.

翻译：论文标题：令牌统计揭示多轮大语言模型交互中的对话漂移 摘要：大语言模型（LLMs）日益部署于多轮交互场景中，其中先前的响应会影响后续响应，因此对话能否在时间上保持一致直接决定其可靠性。当这种一致性未被察觉地退化时，下游决策将失去产生这些决策的对话基础。然而，当前评估方法仅关注孤立输出，而非产生输出的交互过程。本研究证明，对话结构一致性可直接通过令牌频率统计进行监测，无需嵌入、辅助评估器或访问模型内部参数。我们将这一信号形式化为“双向可预测性”（Bipredictability, P），该指标测量上下文、响应及下一轮提示这一循环中共享的可预测性相对于轮次总不确定性的比例，并基于轻量级辅助架构“信息数字孪生”（Information Digital Twin, IDT）实现。在涵盖34种条件的4,574个对话轮次中，基于一个学生模型与三个前沿教师模型，P建立了稳定的运行时基线：在85%的条件下与结构一致性对齐，但仅44%的条件下与语义质量对齐。同时，IDT以100%的灵敏度检测出所有测试中的矛盾、话题偏移及逻辑断裂。这些结果表明，扩展大语言模型交互的可靠性不能仅归结为响应质量，且基于可观测令牌流的结构监测可在部署中补充语义评估。