Large language models (LLMs) have shown promising capabilities to refine their generation based on feedback. However, LLM refinement based on feedback is not always robust and may produce incorrect answers. In this paper, we propose Large LAnguage Model (SALAM) to learn and correct from their mistakes. Our method introduces a study assistant agent to analyze mistakes and generate improvement guidelines from the main LLM. During inference, it identifies common misunderstandings based on the mistake collections and provides guidelines for LLMs to help them avoid similar mistakes. We further finetune the study assistant using imitation learning with successful feedback interaction. Our experiments on two challenging frameworks (BBH and BBQ) demonstrate that SALAM outperforms baselines by a margin of up to 10.7 in accuracy.

翻译：大语言模型（LLMs）展现了基于反馈改进生成结果的显著能力。然而，这种基于反馈的改进并不总是鲁棒的，可能产生错误答案。在本文中，我们提出了大语言模型学习助手（SALAM），旨在使其能够从错误中学习并纠正错误。该方法引入一个学习助手智能体，用于分析主LLM的错误并生成改进指南。在推理阶段，该智能体基于错误集合识别常见的误解，并为LLMs提供指导，帮助其避免同类错误。我们进一步利用成功的反馈交互进行模仿学习，微调学习助手。在两个具有挑战性的框架（BBH和BBQ）上的实验表明，SALAM在准确率上相比基线方法提升了多达10.7个百分点。