Large language models (LLMs) exhibit failure modes on seemingly trivial tasks. We propose a formalisation of LLM interaction using a deterministic multi-tape Turing machine, where each tape represents a distinct component: input characters, tokens, vocabulary, model parameters, activations, probability distributions, and output text. The model enables precise localisation of failure modes to specific pipeline stages, revealing, e.g., how tokenisation obscures character-level structure needed for counting tasks. The model clarifies why techniques like chain-of-thought prompting help, by externalising computation on the output tape, while also revealing their fundamental limitations. This approach provides a rigorous, falsifiable alternative to geometric metaphors and complements empirical scaling laws with principled error analysis.

翻译：大型语言模型（LLMs）在看似简单的任务上表现出特定的失败模式。我们提出了一种基于确定性多带图灵机的LLM交互形式化模型，其中每条带代表一个独立组件：输入字符、词元、词汇表、模型参数、激活值、概率分布和输出文本。该模型能够将失败模式精确定位到流水线的特定阶段，例如揭示词元化过程如何掩盖计数任务所需的字符级结构。该模型阐明了诸如思维链提示等技术为何有效——通过在输出带上外化计算，同时也揭示了其根本局限性。此方法为几何隐喻提供了一种严谨、可证伪的替代方案，并通过原理性错误分析对经验性缩放定律进行了补充。