Generative models lack rigorous statistical guarantees for their outputs and are therefore unreliable in safety-critical applications. In this work, we propose Sequential Conformal Prediction for Generative Models (SCOPE-Gen), a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee called conformal admissibility control. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example. To this end, our method first samples an initial set of i.i.d. examples from a black box generative model. Then, this set is iteratively pruned via so-called greedy filters. As a consequence of the iterative generation procedure, admissibility of the final prediction set factorizes as a Markov chain. This factorization is crucial, because it allows to control each factor separately, using conformal prediction. In comparison to prior work, our method demonstrates a large reduction in the number of admissibility evaluations during calibration. This reduction is important in safety-critical applications, where these evaluations must be conducted manually by domain experts and are therefore costly and time consuming. We highlight the advantages of our method in terms of admissibility evaluations and cardinality of the prediction sets through experiments in natural language generation and molecular graph extension tasks.

翻译：生成模型缺乏对其输出的严格统计保证，因此在安全关键应用中并不可靠。本文提出生成模型的序列保形预测方法（SCOPE-Gen），这是一种序列保形预测方法，能够生成满足严格统计保证（称为保形可接受性控制）的预测集。该保证表明，预测集以高概率至少包含一个可接受（或有效）的样本。为此，我们的方法首先从黑盒生成模型中采样一组独立同分布的初始样本，随后通过所谓的贪婪过滤器对该集合进行迭代剪枝。由于迭代生成过程，最终预测集的可接受性可分解为马尔可夫链。这一分解至关重要，因为它允许使用保形预测分别控制每个因子。与先前工作相比，我们的方法在校准过程中大幅减少了可接受性评估的次数。这种减少在安全关键应用中具有重要意义，因为此类评估必须由领域专家手动执行，成本高昂且耗时。我们通过在自然语言生成和分子图扩展任务中的实验，展示了本方法在可接受性评估次数和预测集基数方面的优势。