This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its average reward under its stationary distribution. Specifically, for each $k=0,\ldots,m-1$ there is a known set ${\mathbb S}_k$ of type-$k$ states. A permissible Markov chain contains exactly one state of each type; the problem is to find a minimum-cost permissible chain. The original motivation was to find a cheapest binary AIFV-$m$ lossless code on a source alphabet of size $n$. Such a code is an $m$-tuple of trees, in which each tree can be viewed as a Markov Chain state. This formulation was then used to address other problems in lossless compression. The known solution techniques for finding minimum-cost Markov chains were iterative and ran in exponential time. This paper shows how to map every possible type-$k$ state into a type-$k$ hyperplane and then define a "Markov Chain Polytope" as the lower envelope of all such hyperplanes. Finding a minimum-cost Markov chain can then be shown to be equivalent to finding a "highest" point on this polytope. The local optimization procedures used in the previous iterative algorithms are shown to be separation oracles for this polytope. Since these were often polynomial time, an application of the Ellipsoid method immediately leads to polynomial time algorithms for these problems.

翻译：本文研究在大型链集合中寻找最小成本$m$状态马尔可夫链$(S_0,\ldots,S_{m-1})$的问题。所研究的链在每个状态上关联着奖励值，链的成本即其"增益"，也就是在平稳分布下的平均奖励。具体而言，对于每个$k=0,\ldots,m-1$，存在已知的类型$k$状态集合${\mathbb S}_k$。一个允许的马尔可夫链需恰好包含每种类型的一个状态；问题在于寻找最小成本的允许链。最初的动机是在大小为$n$的源字母表上寻找最便宜的二进制AIFV-$m$无损编码。此类编码是由$m$棵树组成的元组，其中每棵树可视为马尔可夫链的一个状态。该表述后来被用于解决无损压缩中的其他问题。现有寻找最小成本马尔可夫链的求解技术采用迭代方法且需要指数时间。本文展示了如何将所有可能的类型$k$状态映射到类型$k$超平面，进而将"马尔可夫链多面体"定义为所有这些超平面的下包络。由此可证明，寻找最小成本马尔可夫链等价于在该多面体上寻找"最高"点。先前迭代算法中使用的局部优化过程可视为该多面体的分离预言机。由于这些过程通常具有多项式时间复杂度，应用椭球法即可直接为此类问题导出多项式时间算法。