Language models have emerged as a critical area of focus in artificial intelligence, particularly with the introduction of groundbreaking innovations like ChatGPT. Large-scale Transformer networks have quickly become the leading approach for advancing natural language processing algorithms. Built on the Transformer architecture, these models enable interactions that closely mimic human communication and, equipped with extensive knowledge, can even assist in guiding human tasks. Despite their impressive capabilities and growing complexity, a key question remains-the theoretical foundations of large language models (LLMs). What makes Transformer so effective for powering intelligent language applications, such as translation and coding? What underlies LLMs' ability for In-Context Learning (ICL)? How does the LoRA scheme enhance the fine-tuning of LLMs? And what supports the practicality of pruning LLMs? To address these critical questions and explore the technological strategies within LLMs, we leverage the Universal Approximation Theory (UAT) to offer a theoretical backdrop, shedding light on the mechanisms that underpin these advancements.

翻译：语言模型已成为人工智能领域的关键研究方向，特别是随着ChatGPT等突破性创新的出现。大规模Transformer网络已迅速成为推进自然语言处理算法的主流方法。基于Transformer架构构建的这些模型能够实现高度拟人化的交互，并在海量知识支持下甚至可辅助指导人类任务。尽管其能力卓越且复杂度日益增长，一个核心问题仍悬而未决——大语言模型（LLMs）的理论基础。是什么使Transformer在翻译与代码生成等智能语言应用中如此高效？大语言模型上下文学习能力的本质是什么？LoRA方案如何增强大语言模型的微调效果？又是什么支撑着大语言模型剪枝的可行性？为回答这些关键问题并探索大语言模型内部的技术策略，我们借助通用逼近理论构建理论框架，从而揭示这些技术进步的内在机制。