We introduce two new extensions to the beam search algorithm based on conformal predictions (CP) to produce sets of sequences with theoretical coverage guarantees. The first method is very simple and proposes dynamically-sized subsets of beam search results but, unlike typical CP procedures, has an upper bound on the achievable guarantee depending on a post-hoc calibration measure. Our second algorithm introduces the conformal set prediction procedure as part of the decoding process, producing a variable beam width which adapts to the current uncertainty. While more complex, this procedure can achieve coverage guarantees selected a priori. We provide marginal coverage bounds for each method, and evaluate them empirically on a selection of tasks drawing from natural language processing and chemistry.

翻译：我们基于保形预测（CP）向束搜索算法引入了两种新的扩展，以生成具有理论覆盖保证的序列集合。第一种方法非常简单，它提出了基于束搜索结果的动态大小子集，但与典型的CP流程不同，其可达保证存在上界，该上界取决于事后校准度量。第二种算法将保形集预测过程作为解码的一部分引入，生成了可适应当前不确定性的可变束宽度。虽然更为复杂，但此过程能够实现预先选定的覆盖保证。我们为每种方法提供了边际覆盖界，并在选自自然语言处理和化学领域的若干任务上对其进行了实证评估。