The safe integration of machine learning modules in decision-making processes hinges on their ability to quantify uncertainty. A popular technique to achieve this goal is conformal prediction (CP), which transforms an arbitrary base predictor into a set predictor with coverage guarantees. While CP certifies the predicted set to contain the target quantity with a user-defined tolerance, it does not provide control over the average size of the predicted sets, i.e., over the informativeness of the prediction. In this work, a theoretical connection is established between the generalization properties of the base predictor and the informativeness of the resulting CP prediction sets. To this end, an upper bound is derived on the expected size of the CP set predictor that builds on generalization error bounds for the base predictor. The derived upper bound provides insights into the dependence of the average size of the CP set predictor on the amount of calibration data, the target reliability, and the generalization performance of the base predictor. The theoretical insights are validated using simple numerical regression and classification tasks.

翻译：机器学习模块在决策过程中的安全整合取决于其量化不确定性的能力。实现这一目标的常用技术是保形预测（CP），它可将任意基础预测器转化为具有覆盖保证的集合预测器。尽管CP能确保预测集以用户定义的容错率包含目标量，但它无法控制预测集的平均规模，即预测的信息量。本研究建立了基础预测器泛化特性与CP预测集信息量之间的理论联系。为此，我们推导出CP集合预测器期望规模的上界，该上界基于基础预测器的泛化误差界限。所得上界揭示了CP集合预测器平均规模对校准数据量、目标可靠性及基础预测器泛化性能的依赖规律。通过简单的数值回归与分类任务验证了理论洞见。