Conformal prediction equips machine learning models with a reasonable notion of uncertainty quantification without making strong distributional assumptions. It wraps around any black-box prediction model and converts point predictions into set predictions that have a predefined marginal coverage guarantee. However, conformal prediction only works if we fix the underlying machine learning model in advance. A relatively unaddressed issue in conformal prediction is that of model selection and/or aggregation: for a given problem, which of the plethora of prediction methods (random forests, neural nets, regularized linear models, etc.) should we conformalize? This paper proposes a new approach towards conformal model aggregation in online settings that is based on combining the prediction sets from several algorithms by voting, where weights on the models are adapted over time based on past performance.
翻译:保形预测为机器学习模型提供了一种合理的不确定性量化概念,同时无需做出强分布假设。它能够包裹任何黑箱预测模型,并将点预测转化为具有预定边际覆盖保证的集合预测。然而,保形预测仅在预先固定底层机器学习模型时有效。保形预测中一个相对未被充分解决的问题是模型选择和/或聚合:针对给定问题,面对众多预测方法(随机森林、神经网络、正则化线性模型等),应当对哪一种方法进行保形化?本文提出了一种在线环境下保形模型聚合的新方法,该方法通过投票组合多个算法的预测集合,其中模型的权重会根据过去的表现随时间动态调整。