Incorporating natural language rationales in the prompt and In-Context Learning (ICL) has led to a significant improvement of Large Language Models (LLMs) performance. However, rationales currently require human-annotation or the use of auxiliary proxy models to target promising samples or generate high-quality rationales. In this work, we propose Self-AMPLIFY to generate automatically rationales from post hoc explanation methods applied to Small Language Models (SLMs) to improve their own performance. Self-AMPLIFY is a 3-step method that targets samples, generates rationales and builds a final prompt to leverage ICL. Self-AMPLIFY performance is evaluated on two SLMs and two datasets requiring reasoning abilities: these experiments show that Self-AMPLIFY achieves good results against competitors. Self-AMPLIFY is the first method to apply post hoc explanation methods to SLM to generate rationales to improve their own performance in a fully automated manner.

翻译：将自然语言依据融入提示与上下文学习（ICL）中，已显著提升了大型语言模型（LLM）的性能。然而，当前提供依据仍需人工标注或借助辅助代理模型来筛选优质样本或生成高质量依据。本文提出Self-AMPLIFY方法，通过对小型语言模型（SLM）应用事后解释方法自动生成依据，进而提升其自身性能。Self-AMPLIFY采用三步骤方法：筛选样本、生成依据、构建最终提示以利用ICL。我们在两个SLM及两个需要推理能力的数据集上评估了Self-AMPLIFY性能：实验表明，Self-AMPLIFY相较于竞品取得了优异结果。Self-AMPLIFY是首个将事后解释方法应用于SLM，以完全自动化方式生成依据并提升其自身性能的方法。