LLMs are widely used in complex AI applications. These applications underscore the need for LLM outputs to adhere to a specific format, for their integration with other components in the systems. Typically the format rules e.g., for data serialization formats such as JSON, YAML, or Code in Programming Language are expressed as context-free grammar (CFG). Due to the hallucinations and unreliability of LLMs, instructing LLMs to adhere to specified syntax becomes an increasingly important challenge. We present SynCode, a novel framework for efficient and general syntactical decoding with LLMs, to address this challenge. SynCode ensures soundness and completeness with respect to the CFG of a formal language, effectively retaining valid tokens while filtering out invalid ones. SynCode uses an offline-constructed, efficient lookup table, the DFA mask store, derived from the DFA of the language's grammar for efficient generation. SynCode seamlessly integrates with any language defined by CFG, as evidenced by experiments focusing on generating JSON, Python, and Go outputs. Our experiments evaluating the effectiveness of SynCode for JSON generation demonstrate that SynCode eliminates all syntax errors and significantly outperforms state-of-the-art baselines. Furthermore, our results underscore how SynCode significantly reduces 96.07% of syntax errors in generated Python and Go code, showcasing its substantial impact on enhancing syntactical precision in LLM generation. Our code is available at https://github.com/uiuc-focal-lab/syncode

翻译：大语言模型（LLM）在复杂人工智能应用中得到了广泛使用。这些应用强调LLM输出必须遵循特定格式，以便与系统中的其他组件集成。通常，格式规则（例如用于JSON、YAML等数据序列化格式或编程语言代码）以上下文无关文法（CFG）表示。由于LLM存在幻觉和不可靠性问题，指导LLM遵循指定语法已成为日益重要的挑战。为此，我们提出了SynCode，一个新颖的、高效且通用的LLM语法解码框架。SynCode确保相对于形式语言的CFG具有可靠性和完备性，能有效保留有效标记并过滤无效标记。该框架使用基于语言文法DFA离线构建的高效查找表——DFA掩码存储，以实现高效生成。实验证明，SynCode可无缝集成任何由CFG定义的语言，重点测试了JSON、Python和Go输出的生成。在评估SynCode对JSON生成有效性的实验中，SynCode完全消除了语法错误，并显著优于当前最先进的基线方法。此外，我们的结果表明，SynCode在生成的Python和Go代码中显著减少了96.07%的语法错误，彰显了其在提升LLM生成语法精确性方面的实质性影响。代码已开源：https://github.com/uiuc-focal-lab/syncode