In-context learning (ICL) allows LLMs to learn from examples without changing their weights: this is a particularly promising capability for long-context LLMs that can potentially learn from many examples. Recently, Lin et al. (2024) proposed URIAL, a method using only three in-context examples to align base LLMs, achieving non-trivial instruction following performance. In this work, we show that, while effective, ICL alignment with URIAL still underperforms compared to instruction fine-tuning on the established benchmark MT-Bench, especially with more capable base LLMs. We then uncover the most relevant elements for successful in-context alignment, finding the crucial role of the decoding parameters. Based on these insights, we show that the approach of URIAL can indeed be improved by adding high-quality, potentially carefully selected via greedy search, demonstrations in context, getting closer to the performance of instruct models. Finally, we provide the first, to our knowledge, systematic comparison of ICL and instruction fine-tuning (IFT) for instruction following in the low data regime, where ICL can be a viable alternative to IFT. Overall, our work advances the understanding of ICL as an alignment technique and its relationship to IFT. We provide our code at https://github.com/tml-epfl/icl-alignment.

翻译：上下文学习（ICL）使大语言模型（LLMs）能够在不改变其权重的情况下从示例中学习：对于可能从大量示例中学习的长上下文大语言模型而言，这是一项特别有前景的能力。最近，Lin等人（2024）提出了URIAL方法，该方法仅使用三个上下文示例来对齐基础大语言模型，取得了非平凡的指令遵循性能。在本工作中，我们表明，尽管有效，但URIAL的上下文学习对齐在既定基准MT-Bench上的表现仍逊于指令微调，尤其是在使用能力更强的基础大语言模型时。随后，我们揭示了成功进行上下文对齐的最关键要素，发现了解码参数的核心作用。基于这些见解，我们证明，通过在上下文中添加高质量（可能通过贪婪搜索精心选择）的演示示例，确实可以改进URIAL方法，使其性能更接近指令调优模型。最后，据我们所知，我们首次在低数据场景下对指令遵循任务中的上下文学习与指令微调进行了系统比较，结果表明上下文学习可以成为指令微调的一个可行替代方案。总体而言，我们的工作增进了对上下文学习作为一种对齐技术及其与指令微调关系的理解。我们的代码发布于 https://github.com/tml-epfl/icl-alignment。