We introduce Youtu-LLM, a lightweight yet powerful language model that harmonizes high computational efficiency with native agentic intelligence. Unlike typical small models that rely on distillation, Youtu-LLM (1.96B) is pre-trained from scratch to systematically cultivate reasoning and planning capabilities. The key technical advancements are as follows: (1) Compact Architecture with Long-Context Support: Built on a dense Multi-Latent Attention (MLA) architecture with a novel STEM-oriented vocabulary, Youtu-LLM supports a 128k context window. This design enables robust long-context reasoning and state tracking within a minimal memory footprint, making it ideal for long-horizon agent and reasoning tasks. (2) Principled "Commonsense-STEM-Agent" Curriculum: We curated a massive corpus of approximately 11T tokens and implemented a multi-stage training strategy. By progressively shifting the pre-training data distribution from general commonsense to complex STEM and agentic tasks, we ensure the model acquires deep cognitive abilities rather than superficial alignment. (3) Scalable Agentic Mid-training: Specifically for the agentic mid-training, we employ diverse data construction schemes to synthesize rich and varied trajectories across math, coding, and tool-use domains. This high-quality data enables the model to internalize planning and reflection behaviors effectively. Extensive evaluations show that Youtu-LLM sets a new state-of-the-art for sub-2B LLMs. On general benchmarks, it achieves competitive performance against larger models, while on agent-specific tasks, it significantly surpasses existing SOTA baselines, demonstrating that lightweight models can possess strong intrinsic agentic capabilities.

翻译：我们介绍了Youtu-LLM，一个轻量级但功能强大的语言模型，它实现了高计算效率与原生智能体智能的协调统一。与通常依赖蒸馏的小型模型不同，Youtu-LLM（1.96B）是从头开始预训练的，旨在系统性地培养推理与规划能力。其关键技术进展如下：（1）支持长上下文的紧凑架构：基于密集的多潜在注意力（MLA）架构和一种新颖的STEM导向词表构建，Youtu-LLM支持128k的上下文窗口。该设计使其能够在极小的内存占用下实现鲁棒的长上下文推理与状态追踪，非常适合长视野智能体与推理任务。（2）原则性的“常识-STEM-智能体”课程：我们整理了一个约11T token的大规模语料库，并实施了一个多阶段训练策略。通过将预训练数据分布从通用常识逐步转向复杂的STEM和智能体任务，我们确保模型获得深层的认知能力，而非表面的对齐。（3）可扩展的智能体中期训练：专门针对智能体中期训练，我们采用多样化的数据构建方案，在数学、编程和工具使用领域合成了丰富多样的轨迹。这种高质量数据使模型能够有效地内化规划与反思行为。广泛的评估表明，Youtu-LLM为20亿参数以下的LLM树立了新的性能标杆。在通用基准测试中，它取得了与更大模型相竞争的性能；而在智能体特定任务上，它显著超越了现有的SOTA基线，证明了轻量级模型可以具备强大的内在智能体能力。