Language models have emerged as a critical area of focus in artificial intelligence, particularly with the introduction of groundbreaking innovations like ChatGPT. Large-scale Transformer networks have quickly become the leading approach for advancing natural language processing algorithms. Built on the Transformer architecture, these models enable interactions that closely mimic human communication and, equipped with extensive knowledge, can even assist in guiding human tasks. Despite their impressive capabilities and growing complexity, a key question remains-the theoretical foundations of large language models (LLMs). What makes Transformer so effective for powering intelligent language applications, such as translation and coding? What underlies LLMs' ability for In-Context Learning (ICL)? How does the LoRA scheme enhance the fine-tuning of LLMs? And what supports the practicality of pruning LLMs? To address these critical questions and explore the technological strategies within LLMs, we leverage the Universal Approximation Theory (UAT) to offer a theoretical backdrop, shedding light on the mechanisms that underpin these advancements.

翻译：语言模型已成为人工智能领域的关键焦点，特别是随着ChatGPT等突破性创新的出现。大规模Transformer网络已迅速成为推进自然语言处理算法的主流方法。基于Transformer架构构建的这些模型能够实现高度拟人化的交互，并凭借其海量知识库，甚至能够协助指导人类任务。尽管这些模型展现出卓越能力且复杂度日益增长，但其理论基础——大型语言模型（LLMs）的核心原理——仍存关键疑问：是什么使Transformer在翻译与代码生成等智能语言应用中如此高效？LLMs实现上下文学习（ICL）能力的底层机制是什么？LoRA方案如何增强LLMs的微调效果？又是什么支撑着LLMs剪枝技术的实用性？为探究这些关键问题并剖析LLMs内部的技术策略，我们借助通用逼近理论（UAT）构建理论框架，从而揭示推动这些技术进步的内在机理。