We study one-sided and $α$-correct sequential hypothesis testing for data generated by an ergodic, finite-state Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the alternative corresponds to a disjoint set $Q$. We establish a non-asymptotic instance-dependent lower bound on the expected stopping time of any valid sequential test under the alternative, which is asymptotically tight. Our novel analysis improves the existing lower bounds, which are either asymptotic or provably sub-optimal in this setting. Our lower bound incorporates both the stationary distribution and the transition structure induced by the unknown Markov chain. We further propose an optimal test whose expected stopping time matches this lower bound asymptotically as $α\to 0$. We illustrate the usefulness of our framework through applications to sequential detection of model misspecification in Markov Chain Monte Carlo and to testing structural properties, such as the linearity of transition dynamics, in Markov decision processes. Our findings yield a sharp and general characterization of optimal sequential testing procedures under Markovian dependence.

翻译：我们研究由遍历有限状态马尔可夫链生成数据时的一侧和α-正确序贯假设检验。原假设为未知转移矩阵属于给定随机矩阵集合P，备择假设对应不相交集合Q。我们建立了备择假设下任何有效序贯检验期望停止时间的非渐近实例依赖下界，该下界是渐近紧的。我们的新分析改进了现有在该设定下或为渐近性或被证明为次优的下界。该下界同时考虑了未知马尔可夫链诱导的平稳分布与转移结构。我们进一步提出最优检验，其期望停止时间在α→0时渐近匹配该下界。通过马尔可夫链蒙特卡洛中模型误设的序贯检测应用以及马尔可夫决策过程中转移动力线性性等结构属性检验，我们展示了该框架的实用性。我们的研究结果给出了马尔可夫依赖下最优序贯检验程序的清晰且一般化刻画。