As large language models (LLMs) are increasingly deployed in real-world applications, alignment is no longer governed by a single universal notion of safety or helpfulness, but instead by provider- or application-specific model specifications. These specifications are typically long, structured, and frequently updated, yet existing alignment pipelines lack a systematic mechanism to operationalize them as training signals. In this paper, we propose specification-grounded alignment, a new alignment paradigm that treats provider-authored model specifications as the primary alignment target rather than abstract principles or static benchmarks. To instantiate this paradigm, we introduce SpecAlign, a framework that synthesizes alignment data directly from specification documents. SpecAlign combines structured rule annotation, controllable specification instantiation, and multi-agent adversarial data synthesis to generate fine-grained, boundary-aware preference pairs that capture both compliant behaviors and meaningful specification violations. Experiments across multiple model specifications and backbone models demonstrate that training with SpecAlign consistently improves rule compliance while preserving general capabilities and avoiding over-conservative behavior. These results suggest that grounding alignment in explicit model specifications enables rapid, precise, and scalable adaptation of LLM behavior to evolving policy requirements.

翻译：随着大语言模型（LLMs）在现实应用中的部署日益广泛，对齐不再仅仅由单一、普适的安全性或有用性概念所主导，而是由特定于提供商或应用的模型规格所引导。这些规格通常篇幅较长、结构清晰且频繁更新，然而现有的对齐流程缺乏系统性的机制将其转化为训练信号。本文提出了一种新的对齐范式——面向规格的对齐，将提供商编写的模型规格作为首要的对齐目标，而非抽象的原则或静态的基准。为实例化该范式，我们引入了SpecAlign框架，该框架直接从规格文档中合成对齐数据。SpecAlign结合了结构化规则标注、可控规格实例化以及多智能体对抗性数据合成，生成既包含合规行为又包含有意义规格违规的细粒度、边界感知偏好对。在多种模型规格和骨干模型上的实验表明，使用SpecAlign进行训练能够持续提升规则遵循能力，同时保持通用能力并避免过度保守的行为。这些结果表明，将对齐基于显式的模型规格能够实现大语言模型行为对不断演变的策略要求的快速、精准且可扩展的适应。