Consider the problem of predicting the next symbol given a sample path of length n, whose joint distribution belongs to a distribution class that may have long-term memory. The goal is to compete with the conditional predictor that knows the true model. For both hidden Markov models (HMMs) and renewal processes, we determine the optimal prediction risk in Kullback- Leibler divergence up to universal constant factors. Extending existing results in finite-order Markov models [HJW23] and drawing ideas from universal compression, the proposed estimator has a prediction risk bounded by redundancy of the distribution class and a memory term that accounts for the long-range dependency of the model. Notably, for HMMs with bounded state and observation spaces, a polynomial-time estimator based on dynamic programming is shown to achieve the optimal prediction risk {\Theta}(log n/n); prior to this work, the only known result of this type is O(1/log n) obtained using Markov approximation [Sha+18]. Matching minimax lower bounds are obtained by making connections to redundancy and mutual information via a reduction argument.

翻译：考虑根据长度为n的样本路径预测下一个符号的问题，该路径的联合分布可能属于具有长期记忆的分布类别。目标是竞争已知真实模型的条件预测器。对于隐马尔可夫模型与更新过程，我们确定了在Kullback-Leibler散度下达到通用常数因子最优的预测风险。通过扩展有限阶马尔可夫模型[HJW23]的现有结果并借鉴通用压缩思想，所提出的估计器的预测风险受限于分布类别的冗余度与反映模型长程依赖性的记忆项。值得注意的是，对于状态空间和观测空间有界的隐马尔可夫模型，基于动态规划的多项式时间估计器被证明可达到最优预测风险{\Theta}(log n/n)；在此工作之前，此类结果的唯一已知结论是采用马尔可夫逼近[Sha+18]获得的O(1/log n)。通过归约论证将最优性下界与冗余度和互信息相关联，我们得到了匹配的极小化极大下界。