Large Language Models (LLMs) have benefited enormously from scaling, yet these gains are bounded by five fundamental limitations: (1) hallucination, (2) context compression, (3) reasoning degradation, (4) retrieval fragility, and (5) multimodal misalignment. While existing surveys describe these phenomena empirically, they lack a rigorous theoretical synthesis connecting them to the foundational limits of computation, information, and learning. This work closes that gap by presenting a unified, proof-informed framework that formalizes the innate theoretical ceilings of LLM scaling. First, computability and uncomputability imply an irreducible residue of error: for any computably enumerable model family, diagonalization guarantees inputs on which some model must fail, and undecidable queries (e.g., halting-style tasks) induce infinite failure sets for all computable predictors. Second, information-theoretic and statistical constraints bound attainable accuracy even on decidable tasks, finite description length enforces compression error, and long-tail factual knowledge requires prohibitive sample complexity. Third, geometric and computational effects compress long contexts far below their nominal size due to positional under-training, encoding attenuation, and softmax crowding. We further show how likelihood-based training favors pattern completion over inference, how retrieval under token limits suffers from semantic drift and coupling noise, and how multimodal scaling inherits shallow cross-modal alignment. Across sections, we pair theorems and empirical evidence to outline where scaling helps, where it saturates, and where it cannot progress, providing both theoretical foundations and practical mitigation paths like bounded-oracle retrieval, positional curricula, and sparse or hierarchical attention.

翻译：大规模语言模型（LLMs）从模型扩展中获益巨大，然而这些增益受到五个基本限制的约束：（1）幻觉，（2）上下文压缩，（3）推理能力退化，（4）检索脆弱性，以及（5）多模态错位。虽然现有的综述从经验上描述了这些现象，但它们缺乏一个严谨的理论综合，将这些现象与计算、信息和学习的基本极限联系起来。本研究通过提出一个统一的、基于证明的框架来弥补这一差距，该框架形式化了LLM扩展的内在理论上限。首先，可计算性与不可计算性意味着一个不可约的错误残余：对于任何可计算枚举的模型族，对角化保证了存在某些输入，在这些输入上某些模型必然失败，而不可判定查询（例如，停机类任务）会为所有可计算预测器诱导出无限的失败集。其次，信息论和统计约束限制了即使在可判定任务上可达到的准确性，有限描述长度强制了压缩误差，而长尾事实知识需要过高的样本复杂度。第三，几何和计算效应将长上下文压缩到远低于其名义规模，这是由于位置训练不足、编码衰减和softmax拥挤所致。我们进一步展示了基于似然的训练如何偏爱模式补全而非推理，在令牌限制下的检索如何遭受语义漂移和耦合噪声，以及多模态扩展如何继承浅层的跨模态对齐。在各个部分中，我们结合定理和实证证据，勾勒出扩展在何处有益、在何处饱和以及在何处无法推进，同时提供了理论基础和实际缓解路径，例如有界预言检索、位置课程学习以及稀疏或分层注意力机制。