Constrained decoding, a technique for enforcing constraints on language model outputs, offers a way to control text generation without retraining or architectural modifications. Its application is, however, typically restricted to models that give users access to next-token distributions (usually via softmax logits), which poses a limitation with blackbox large language models (LLMs). This paper introduces sketch-guided constrained decoding (SGCD), a novel approach to constrained decoding for blackbox LLMs, which operates without access to the logits of the blackbox LLM. SGCD utilizes a locally hosted auxiliary model to refine the output of an unconstrained blackbox LLM, effectively treating this initial output as a "sketch" for further elaboration. This approach is complementary to traditional logit-based techniques and enables the application of constrained decoding in settings where full model transparency is unavailable. We demonstrate the efficacy of SGCD through experiments in closed information extraction and constituency parsing, showing how it enhances the utility and flexibility of blackbox LLMs for complex NLP tasks.

翻译：约束解码是一种对语言模型输出施加约束的技术，它提供了一种无需重新训练或修改架构即可控制文本生成的方法。然而，该技术的应用通常仅限于允许用户访问下一个词元分布（通常通过softmax logits）的模型，这对黑盒大语言模型构成了限制。本文提出了草图引导约束解码，这是一种适用于黑盒大语言模型的新型约束解码方法，其运行无需访问黑盒大语言模型的logits。SGCD利用本地托管的辅助模型来优化无约束黑盒大语言模型的输出，有效地将此初始输出视为进一步细化的"草图"。该方法与传统基于logit的技术互补，使得在缺乏完整模型透明度的场景中应用约束解码成为可能。我们通过在封闭信息抽取和成分句法解析任务上的实验证明了SGCD的有效性，展示了它如何提升黑盒大语言模型在复杂NLP任务中的实用性和灵活性。