In-context learning (ICL) allows LLMs to learn from examples without changing their weights, which is a particularly promising capability for long-context LLMs that can potentially learn from many examples. Recently, Lin et al. (2024) proposed URIAL, a method using only three in-context examples to align base LLMs, achieving non-trivial instruction following performance. In this work, we show that, while effective, ICL alignment with URIAL still underperforms compared to instruction fine-tuning on established benchmarks such as MT-Bench and AlpacaEval 2.0 (LC), especially with more capable base LMs. Unlike for tasks such as classification, translation, or summarization, adding more ICL demonstrations for long-context LLMs does not systematically improve instruction following performance. To address this limitation, we derive a greedy selection approach for ICL examples that noticeably improves performance, yet without bridging the gap to instruction fine-tuning. Finally, we provide a series of ablation studies to better understand the reasons behind the remaining gap, and we show how some aspects of ICL depart from the existing knowledge and are specific to the instruction tuning setting. Overall, our work advances the understanding of ICL as an alignment technique. We provide our code at https://github.com/tml-epfl/icl-alignment.

翻译：上下文学习（ICL）使大语言模型（LLMs）能够在不改变其权重的情况下从示例中学习，这对于可能从大量示例中学习的长上下文大语言模型而言是一种特别有前景的能力。最近，Lin等人（2024）提出了URIAL方法，该方法仅使用三个上下文示例来对齐基础大语言模型，取得了显著的指令遵循性能。在本研究中，我们表明，尽管有效，但与在MT-Bench和AlpacaEval 2.0（LC）等成熟基准测试上的指令微调相比，使用URIAL进行的上下文学习对齐仍然表现不足，尤其是在使用能力更强的基础大语言模型时。与分类、翻译或摘要等任务不同，为长上下文大语言模型添加更多上下文学习演示示例并不能系统性地提升指令遵循性能。为了解决这一局限，我们推导出一种用于上下文学习示例的贪心选择方法，该方法显著提升了性能，但仍未弥合与指令微调之间的差距。最后，我们进行了一系列消融研究，以更好地理解剩余差距背后的原因，并展示了上下文学习的某些方面如何偏离现有认知，且特定于指令调优场景。总体而言，我们的工作推进了将上下文学习作为一种对齐技术的理解。我们的代码发布于 https://github.com/tml-epfl/icl-alignment。