Question answering over hybrid contexts is a complex task, which requires the combination of information extracted from unstructured texts and structured tables in various ways. Recently, In-Context Learning demonstrated significant performance advances for reasoning tasks. In this paradigm, a large language model performs predictions based on a small set of supporting exemplars. The performance of In-Context Learning depends heavily on the selection procedure of the supporting exemplars, particularly in the case of HybridQA, where considering the diversity of reasoning chains and the large size of the hybrid contexts becomes crucial. In this work, we present Selection of ExEmplars for hybrid Reasoning (SEER), a novel method for selecting a set of exemplars that is both representative and diverse. The key novelty of SEER is that it formulates exemplar selection as a Knapsack Integer Linear Program. The Knapsack framework provides the flexibility to incorporate diversity constraints that prioritize exemplars with desirable attributes, and capacity constraints that ensure that the prompt size respects the provided capacity budgets. The effectiveness of SEER is demonstrated on FinQA and TAT-QA, two real-world benchmarks for HybridQA, where it outperforms previous exemplar selection methods.

翻译：混合情境下的问答是一个复杂任务，需要以多种方式结合从非结构化文本和结构化表格中提取的信息。近年来，情境学习在推理任务中展现出显著的性能提升。在该范式下，大型语言模型基于少量支持性示例执行预测。情境学习的性能高度依赖支持性示例的选择过程，尤其在混合情境问答中，考虑推理链的多样性及混合情境的庞大容量变得至关重要。本文提出混合推理示例选择方法（SEER），这是一种新颖的示例选择方法，能够同时兼顾代表性与多样性。SEER的关键创新在于将示例选择形式化为一个背包整数线性规划问题。该背包框架的灵活性使其能够纳入多样性约束（优先选择具有理想属性的示例）和容量约束（确保提示的大小符合给定的容量预算）。在FinQA和TAT-QA这两个混合情境问答的基准数据集上，SEER展示了其有效性，性能优于以往的示例选择方法。