LLMs are widely used in complex AI applications. These applications underscore the need for LLM outputs to adhere to a specific format, for their integration with other components in the systems. Typically the format rules e.g., for data serialization formats such as JSON, YAML, or Code in Programming Language are expressed as context-free grammar (CFG). Due to the hallucinations and unreliability of LLMs, instructing LLMs to adhere to specified syntax becomes an increasingly important challenge. We present SynCode, a novel framework for efficient and general syntactical decoding with LLMs, to address this challenge. SynCode ensures soundness and completeness with respect to the CFG of a formal language, effectively retaining valid tokens while filtering out invalid ones. SynCode uses an offline-constructed, efficient lookup table, the DFA mask store, derived from the DFA of the language's grammar for efficient generation. SynCode seamlessly integrates with any language defined by CFG, as evidenced by experiments focusing on generating JSON, Python, and Go outputs. Our experiments evaluating the effectiveness of SynCode for JSON generation demonstrate that SynCode eliminates all syntax errors and significantly outperforms state-of-the-art baselines. Furthermore, our results underscore how SynCode significantly reduces 96.07% of syntax errors in generated Python and Go code, showcasing its substantial impact on enhancing syntactical precision in LLM generation. Our code is available at https://github.com/uiuc-focal-lab/syncode

翻译：大语言模型（LLM）已广泛应用于复杂的人工智能系统中。这些应用场景要求LLM的输出必须遵循特定格式，以便与系统中的其他组件进行集成。通常，格式规则（例如用于JSON、YAML等数据序列化格式或编程语言代码）以上下文无关文法（CFG）表示。由于LLM存在幻觉问题且输出不可靠，指导LLM遵循指定语法已成为日益重要的挑战。为此，我们提出SynCode——一个新颖的、高效且通用的LLM语法解码框架。SynCode能确保生成结果相对于形式语言的CFG具备可靠性与完备性，有效保留合法标记并滤除非法标记。该框架采用基于语法确定性有限自动机（DFA）离线构建的高效查询表（DFA掩码存储）来优化生成过程。实验证明，SynCode可无缝适配任何由CFG定义的语言，本文重点展示了其在生成JSON、Python和Go代码上的应用。针对JSON生成的实验表明，SynCode能完全消除语法错误，并显著优于现有先进基线方法。此外，实验结果凸显了SynCode在生成Python和Go代码时能减少96.07%的语法错误，显著提升了LLM生成的语法准确性。代码已开源：https://github.com/uiuc-focal-lab/syncode