Language models, especially pre-trained large language models, have showcased remarkable abilities as few-shot in-context learners (ICL), adept at adapting to new tasks with just a few demonstrations in the input context. However, the model's ability to perform ICL is sensitive to the choice of the few-shot demonstrations. Instead of using a fixed set of demonstrations, one recent development is to retrieve demonstrations tailored to each input query. The implementation of demonstration retrieval is relatively straightforward, leveraging existing databases and retrieval systems. This not only improves the efficiency and scalability of the learning process but also has been shown to reduce biases inherent in manual example selection. In light of the encouraging results and growing research in ICL with retrieved demonstrations, we conduct an extensive review of studies in this area. In this survey, we discuss and compare different design choices for retrieval models, retrieval training procedures, and inference algorithms.

翻译：语言模型，特别是预训练的大规模语言模型，已展现出作为少样本情境学习器的卓越能力，能通过输入上下文中的少量示例灵活适应新任务。然而，模型执行情境学习的能力对少样本示例的选择高度敏感。与使用固定示例集不同，近期的一项进展是为每个输入查询检索定制化的示例。示例检索的实现相对直接，可利用现有数据库与检索系统。这不仅提升了学习过程的效率与可扩展性，更被证明能减少人工示例选择中固有的偏差。鉴于检索增强型情境学习领域的显著成果与日益增长的研究兴趣，我们对相关文献进行了全面梳理。本综述将讨论并比较检索模型的不同设计选择、检索训练流程及推理算法。