Uncertainty quantification (UQ) is critical for the deployment of machine learning predictors in real-world scenarios where the data distribution may shift over time (i.e., data may not be exchangeable). Online conformal prediction (OCP) methods address this issue at the expense of either (i) group-wise error control or (ii) learning-rate independent implementation. Group-conditional coverage is essential for fairness across different collections of data points and for providing finer UQ guarantees. Parameter-free optimization is crucial for robustness to adversarial and unknown data shifts. We propose a parameter-free algorithm for group-conditional OCP and demonstrate that it achieves the best group-conditional coverage guarantees. We evaluate our algorithm on synthetic and real-world data, demonstrating that our method not only improves the reliability of existing parameter-free OCP methods but also provides prediction intervals that are comparable in size to well-tuned group-conditional approaches. By unifying group-conditional coverage with parameter-free online algorithms, our work lays a foundation for fair and robust uncertainty quantification in shifting environments.
翻译:不确定性量化(UQ)对于机器学习预测器在数据分布可能随时间变化(即数据可能不可交换)的真实场景中的部署至关重要。在线共形预测(OCP)方法在解决此问题时会牺牲以下两者之一:(i)分组误差控制,或(ii)与学习率无关的实现。分组条件覆盖对于确保不同数据点集合之间的公平性以及提供更精细的UQ保证至关重要。无参数优化对于对抗性及未知数据漂移的鲁棒性具有关键意义。我们提出了一种用于分组条件OCP的无参数算法,并证明该算法能够实现最优的分组条件覆盖保证。通过在合成数据与真实数据上的实验评估,我们证明了该方法不仅提升了现有无参数OCP方法的可靠性,还提供了与经过精细调参的分组条件方法尺寸相当预测区间。通过将分组条件覆盖与无参数在线算法相结合,我们的工作为漂移环境下的公平且鲁棒的不确定性量化奠定了基础。