Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce $\texttt{SEM-CTRL}$, a unified approach that enforces rich context-sensitive constraints and task- and instance-specific semantics directly on an LLM decoder. Our approach integrates token-level MCTS, which is guided by specific syntactic and semantic constraints. The constraints over the desired outputs are expressed using Answer Set Grammars -- a logic-based formalism that generalizes context-sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach guarantees correct completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate $\texttt{SEM-CTRL}$ on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, and planning. Our results demonstrate that $\texttt{SEM-CTRL}$ allows small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models (e.g., o1-preview) while simultaneously guaranteeing solution correctness.

翻译：确保大型语言模型（LLM）输出在句法和语义上的正确性，对于其实际部署至关重要，但这仍然是一个重大挑战。本文提出$\texttt{SEM-CTRL}$，这是一种统一的方法，可直接在LLM解码器上强制执行丰富的上下文敏感约束以及任务和实例特定的语义。我们的方法集成了由特定句法和语义约束引导的令牌级蒙特卡洛树搜索（MCTS）。对期望输出的约束使用答案集语法（Answer Set Grammars）来表达——这是一种基于逻辑的形式化方法，它泛化了上下文敏感语法，同时结合背景知识来表示任务特定的语义。我们证明，我们的方法能够保证任何现成的LLM无需微调即可生成正确的补全。我们在包括合成语法生成、组合推理和规划在内的一系列任务上评估了$\texttt{SEM-CTRL}$。结果表明，$\texttt{SEM-CTRL}$使得小型预训练LLM能够高效地超越更大的模型变体和最先进的推理模型（例如o1-preview），同时保证解决方案的正确性。