Conformal prediction is a widely used method to quantify the uncertainty of a classifier under the assumption of exchangeability (e.g., IID data). We generalize conformal prediction to the Hidden Markov Model (HMM) framework where the assumption of exchangeability is not valid. The key idea of the proposed method is to partition the non-exchangeable Markovian data from the HMM into exchangeable blocks by exploiting the de Finetti's Theorem for Markov Chains discovered by Diaconis and Freedman (1980). The permutations of the exchangeable blocks are viewed as randomizations of the observed Markovian data from the HMM. The proposed method provably retains all desirable theoretical guarantees offered by the classical conformal prediction framework in both exchangeable and Markovian settings. In particular, while the lack of exchangeability introduced by Markovian samples constitutes a violation of a crucial assumption for classical conformal prediction, the proposed method views it as an advantage that can be exploited to improve the performance further. Detailed numerical and empirical results that complement the theoretical conclusions are provided to illustrate the practical feasibility of the proposed method.
翻译:共形预测是一种广泛使用的方法,用于在可交换性假设(如独立同分布数据)下量化分类器的不确定性。我们将其推广到隐马尔可夫模型框架中,在该框架下可交换性假设不再成立。所提出方法的核心思想是,利用Diaconis与Freedman(1980)发现的马尔可夫链de Finetti定理,将HMM中非可交换的马尔可夫数据划分为可交换的区块。这些可交换区块的排列被视为对观测到的HMM马尔可夫数据的随机化处理。所提出方法在可交换与马尔可夫设定下均能证明保留经典共形预测框架的所有理想理论保证。特别地,尽管马尔可夫样本引入的非可交换性违反了经典共形预测的关键假设,但所提方法将其视为可进一步提升性能的优势。本文提供了详细的数值与实证结果以补充理论结论,论证了该方法的实际可行性。