We present a pipeline for building and aggregating task-specific, LLM-generated weak (imperfect) verifiers into a strong verifier for spatial layout domains. Given a task description, our pipeline asks an LLM to synthesize a collection of verifier programs using a layout verification DSL. Each individual LLM-generated verifier usually provides an imperfect check for a match between the layout and the corresponding task description. We show that by aggregating the responses of many such verifiers we can produce a stronger verifier. Moreover, by applying techniques from weak learning, our pipeline can learn how to aggregate the weak verifiers from a very sparse set of human labeled example layouts (about 10). We find that the strong verifiers produced by our pipeline outperform the status-quo approach of using a set of LLM judges to directly check whether a layout matches a task description, raising F1-scores by up to 7X across a variety of 3D room layout and 2D poster design tasks. We also demonstrate that verifier-guided layout generation using natural language feedback from our strong verifiers improves layout quality of a base layout generator by up to 66.2% according to a human evaluator.

翻译：我们提出了一种流水线方法，用于构建并聚合任务特定的、由LLM生成的弱（不完美）验证器，进而形成适用于空间布局领域的强验证器。给定任务描述，该流水线要求LLM利用布局验证领域特定语言（DSL）合成一组验证器程序。每个独立的LLM生成的验证器通常只能对布局与对应任务描述的匹配性提供不完美的检查。研究表明，通过聚合多个此类验证器的响应，我们可以构建更强的验证器。此外，通过应用弱学习技术，该流水线能够依据极其稀疏的人类标注示例布局（约10个）学习如何聚合弱验证器。实验发现，该流水线生成的强验证器在性能上超越了当前主流的直接利用LLM评判集检查布局与任务描述匹配性的方法，在多种3D房间布局和2D海报设计任务中F1分数提升高达7倍。我们还证明，基于强验证器生成的包含自然语言反馈的验证导向布局生成方法，可使基础布局生成器的布局质量据人类评估者评价提升最高达66.2%。