We address the fundamental limits of learning unknown parameters of any stochastic process from time-series data, and discover exact closed-form expressions for how optimal inference scales with observation length. Given a parametrized class of candidate models, the Fisher information of observed sequence probabilities lower-bounds the variance in model estimation from finite data. As sequence-length increases, the minimal variance scales as the square inverse of the length -- with constant coefficient given by the information rate. We discover a simple closed-form expression for this information rate, even in the case of infinite Markov order. We furthermore obtain the exact analytic lower bound on model variance from the observation-induced metadynamic among belief states. We discover ephemeral, exponential, and more general modes of convergence to the asymptotic information rate. Surprisingly, this myopic information rate converges to the asymptotic Fisher information rate with exactly the same relaxation timescales that appear in the myopic entropy rate as it converges to the Shannon entropy rate for the process. We illustrate these results with a sequence of examples that highlight qualitatively distinct features of stochastic processes that shape optimal learning.

翻译：我们从时间序列数据中学习任意随机过程未知参数的基本极限出发，发现了最优推断随观测长度变化规律的精确闭式表达式。给定参数化的候选模型类别，观测序列概率的费舍尔信息对有限数据下模型估计的方差打下界。随着序列长度增加，最小方差以长度平方反比缩放——其常系数由信息率给出。我们发现了该信息率的简洁闭式表达式，即使在无限马尔可夫阶情况下也成立。此外，我们还从观测引发的信念状态间元动力学中，得到了模型方差的精确解析下界。我们发现了渐近信息率收敛中的瞬态、指数及更一般模式。令人惊讶的是，这种短视信息率收敛到渐近费舍尔信息率时，其松弛时间尺度与短视熵率收敛到过程香农熵率时出现的松弛时间尺度完全相同。我们通过一系列例子阐明这些结果，这些例子凸显了塑造最优学习的随机过程的定性不同特征。