Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce $\texttt{SEM-CTRL}$, a unified approach that allows for enforcing rich context-sensitive constraints, and task and instance specific semantics directly on the LLM decoder. Our approach integrates token-level MCTS which is guided by specific syntactic and semantic constraints. The constraints over desired outputs are expressed using Answer Set Grammars, which is a logic-based formalism that generalizes context sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach helps guarantee valid completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate $\texttt{SEM-CTRL}$ on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, JSON parsing, and planning. Our experimental results demonstrate that $\texttt{SEM-CTRL}$ allows even small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models (e.g., $\textit{o4-mini}$) while simultaneously guaranteeing semantic validity.

翻译：确保大型语言模型（LLM）输出在语法和语义上的正确性仍是重大挑战，尽管这对实际部署至关重要。本文提出统一方法 $\texttt{SEM-CTRL}$，可直接在LLM解码器中施加丰富的上下文敏感约束、任务及实例特定语义。该方法整合了由特定语法和语义约束引导的标记级蒙特卡洛树搜索（MCTS）。对期望输出的约束通过答案集语法（Answer Set Grammars）表达，这是一种基于逻辑的形式化方法，在泛化上下文敏感语法的同时融入背景知识以表示任务特定语义。我们表明，该方法有助于保证任何现成LLM生成有效补全结果，且无需微调。我们在包括合成语法生成、组合推理、JSON解析和规划在内的多项任务上评估了 $\texttt{SEM-CTRL}$。实验结果表明，$\texttt{SEM-CTRL}$ 能使小型预训练LLM在保证语义有效性的同时，高效超越更大规模变体及最先进的推理模型（如 $\textit{o4-mini}$）。