Natural generation allows Language Models (LMs) to produce free-form responses with rich reasoning, but the lack of guaranteed structure makes outputs difficult to parse or verify. Structured generation, or constrained decoding, addresses this drawback by producing content in standardized formats such as JSON, ensuring consistency and guaranteed-parsable outputs, but it can inadvertently restrict the model's reasoning capabilities. In this work, we propose a simple approach that combines the advantages of both natural and structured generation. By allowing LLMs to reason freely until specific trigger tokens are generated, and then switching to structured generation, our method preserves the expressive power of natural language reasoning while ensuring the reliability of structured outputs. We further evaluate our approach on several datasets, covering both classification and reasoning tasks, to demonstrate its effectiveness, achieving a substantial gain of up to 27% in accuracy compared to natural generation, while requiring only a small overhead of 10-20 extra tokens.

翻译：自然生成允许语言模型（LM）产生具有丰富推理的自由形式响应，但缺乏保证的结构使得输出难以解析或验证。结构化生成（或称约束解码）通过以标准化格式（如JSON）生成内容来解决这一缺点，确保一致性和可解析的输出，但它可能无意中限制模型的推理能力。本文提出一种结合自然生成与结构化生成优势的简单方法。通过允许大语言模型在生成特定触发标记前自由推理，随后切换至结构化生成，我们的方法既保留了自然语言推理的表达能力，又确保了结构化输出的可靠性。我们在多个数据集上进一步评估了该方法，涵盖分类与推理任务，以证明其有效性：相比自然生成实现了高达27%的准确率提升，而仅需10-20个额外标记的微小开销。