We present an algorithm for learning mixtures of Markov chains and Markov decision processes (MDPs) from short unlabeled trajectories. Specifically, our method handles mixtures of Markov chains with optional control input by going through a multi-step process, involving (1) a subspace estimation step, (2) spectral clustering of trajectories using "pairwise distance estimators," along with refinement using the EM algorithm, (3) a model estimation step, and (4) a classification step for predicting labels of new trajectories. We provide end-to-end performance guarantees, where we only explicitly require the length of trajectories to be linear in the number of states and the number of trajectories to be linear in a mixing time parameter. Experimental results support these guarantees, where we attain 96.6% average accuracy on a mixture of two MDPs in gridworld, outperforming the EM algorithm with random initialization (73.2% average accuracy).

翻译：本文提出了一种算法，用于从短无标签轨迹中学习马尔可夫链与马尔可夫决策过程（MDPs）的混合模型。具体地，我们的方法通过多步骤流程处理带有可选控制输入的马尔可夫链混合模型，包括：（1）子空间估计步骤，（2）利用“成对距离估计器”对轨迹进行谱聚类，并结合期望最大化（EM）算法进行优化，（3）模型估计步骤，以及（4）预测新轨迹标签的分类步骤。我们提供了端到端的性能保证，仅需轨迹长度与状态数呈线性关系，且轨迹数量与混合时间参数呈线性关系。实验结果支持这些保证：在网格世界中两个MDP的混合任务中，我们实现了96.6%的平均准确率，优于随机初始化的EM算法（平均准确率73.2%）。