Previous in-context learning (ICL) research has focused on tasks such as classification, machine translation, text2table, etc., while studies on whether ICL can improve human-like dialogue generation are scarce. Our work fills this gap by systematically investigating the ICL capabilities of large language models (LLMs) in persona-based dialogue generation, conducting extensive experiments on high-quality real human Chinese dialogue datasets. From experimental results, we draw three conclusions: 1) adjusting prompt instructions is the most direct, effective, and economical way to improve generation quality; 2) randomly retrieving demonstrations (demos) achieves the best results, possibly due to the greater diversity and the amount of effective information; counter-intuitively, retrieving demos with a context identical to the query performs the worst; 3) even when we destroy the multi-turn associations and single-turn semantics in the demos, increasing the number of demos still improves dialogue performance, proving that LLMs can learn from corrupted dialogue demos. Previous explanations of the ICL mechanism, such as $n$-gram induction head, cannot fully account for this phenomenon.

翻译：既往的上下文学习研究主要集中在分类、机器翻译、文本到表格等任务上，而关于上下文学习能否改善类人对话生成的研究却很少。我们的工作通过系统探究大语言模型在基于人格的对话生成中的上下文学习能力，并在高质量真实人类中文对话数据集上进行广泛实验，填补了这一空白。根据实验结果，我们得出三个结论：1）调整提示指令是提升生成质量最直接、有效且经济的方式；2）随机检索示例能达到最佳效果，这可能是由于示例的多样性和有效信息量更大；与之相反的是，检索与查询上下文完全相同的示例效果最差；3）即使我们破坏了示例中的多轮关联性和单轮语义，增加示例数量仍能提升对话性能，证明大语言模型可以从受损的对话示例中学习。以往对上下文学习机制的解释（如n-gram归纳头）无法完全解释这一现象。