Large language models (LLMs) have shown great abilities of solving various natural language tasks in different domains. Due to the training objective of LLMs and their pre-training data, LLMs are not very well equipped for tasks involving structured data generation. We propose a framework, Prompting with Iterative Verification (PiVe), to improve graph-based generative capability of LLMs. We show how a small language model could be trained to act as a verifier module for the output of an LLM~(i.e., ChatGPT, GPT-4), and to iteratively improve its performance via fine-grained corrective instructions. We also show how the verifier module could apply iterative corrections offline for a more cost-effective solution to the text-to-graph generation task. Experiments on three graph-based datasets show consistent improvement gained via PiVe. Additionally, we create GenWiki-HIQ and highlight that the verifier module can be used as a data augmentation tool to help improve the quality of automatically generated parallel text-graph datasets.

翻译：大型语言模型（LLM）在不同领域的多种自然语言处理任务中展现出卓越能力。然而，受其训练目标与预训练数据的限制，LLM在处理涉及结构化数据生成的任务时表现尚不完善。本文提出一种基于迭代验证的提示框架（PiVe），旨在提升LLM的图结构生成能力。我们展示了如何训练一个小型语言模型作为LLM（如ChatGPT、GPT-4）输出的验证模块，并通过细粒度纠错指令迭代优化其性能。同时，我们验证了该模块可离线执行迭代修正，为文本到图生成任务提供更具成本效益的解决方案。在三个图结构数据集上的实验表明，PiVe框架能带来持续的性能提升。此外，我们构建了GenWiki-HIQ数据集，并证明验证模块可作为数据增强工具，有效提升自动生成的并行文本-图数据集的质量。