Like people, LLMs do not always generate the best text for a given generation problem on their first try (e.g., summaries, answers, explanations). Just as people then refine their text, we introduce SELF-REFINE, a framework for similarly improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an output using an LLM, then allow the same model to provide multi-aspect feedback for its own output; finally, the same model refines its previously generated output given its own feedback. Unlike earlier work, our iterative refinement framework does not require supervised training data or reinforcement learning, and works with a single LLM. We experiment with 7 diverse tasks, ranging from review rewriting to math reasoning, demonstrating that our approach outperforms direct generation. In all tasks, outputs generated with SELF-REFINE are preferred by humans and by automated metrics over those generated directly with GPT-3.5 and GPT-4, improving on average by absolute 20% across tasks.

翻译：如同人类一样，大语言模型（LLMs）在处理特定生成任务（如摘要生成、问答、解释说明）时，首次尝试生成的文本往往并非最优。受人类反复修改文本的启发，我们提出了SELF-REFINE框架——一种通过迭代反馈和优化来改进LLM初始输出的方法。其核心思想是：先利用LLM生成初始输出，再让同一模型对其自身输出进行多维度反馈，最后基于该反馈对先前输出进行优化。与早期研究不同，本迭代优化框架无需监督训练数据或强化学习，仅需单一LLM即可实现。我们在涵盖评论改写至数学推理的7个多样化任务中进行了实验，结果表明该方法显著优于直接生成策略。在所有任务中，无论是人类评测还是自动化指标均更偏好采用SELF-REFINE生成的输出，相较于直接使用GPT-3.5和GPT-4生成的方案，各项任务指标平均绝对提升达20%。