This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its average reward under its stationary distribution. Specifically, for each $k=0,\ldots,m-1$ there is a known set ${\mathbb S}_k$ of type-$k$ states. A permissible Markov chain contains exactly one state of each type; the problem is to find a minimum-cost permissible chain. The original motivation was to find a cheapest binary AIFV-$m$ lossless code on a source alphabet of size $n$. Such a code is an $m$-tuple of trees, in which each tree can be viewed as a Markov Chain state. This formulation was then used to address other problems in lossless compression. The known solution techniques for finding minimum-cost Markov chains were iterative and ran in exponential time. This paper shows how to map every possible type-$k$ state into a type-$k$ hyperplane and then define a "Markov Chain Polytope" as the lower envelope of all such hyperplanes. Finding a minimum-cost Markov chain can then be shown to be equivalent to finding a "highest" point on this polytope. The local optimization procedures used in the previous iterative algorithms are shown to be separation oracles for this polytope. Since these were often polynomial time, an application of the Ellipsoid method immediately leads to polynomial time algorithms for these problems.

翻译：本文研究在大量链中寻找最小成本的$m$状态马尔可夫链$(S_0,\ldots,S_{m-1})$问题。所研究的链中每个状态关联一个奖励值，链的成本定义为"增益"，即在其平稳分布下的平均奖励。具体而言，对于每个$k=0,\ldots,m-1$，存在已知的$k$型状态集${\mathbb S}_k$。允许的马尔可夫链恰好包含每个类型的一个状态，问题在于寻找最小成本的允许链。最初动机是寻找源字母表大小为$n$的最廉价二元AIFV-$m$无损码。这种码由$m$个树构成的元组表示，其中每个树可视为马尔可夫链状态。该公式后来被用于解决无损压缩中的其他问题。已知寻找最小成本马尔可夫链的求解技术采用迭代方法且运行时间为指数级。本文展示了如何将每个可能的$k$型状态映射到$k$型超平面，进而将"马尔可夫链多面体"定义为所有此类超平面的下包络。于是，寻找最小成本马尔可夫链等价于在该多面体上寻找"最高"点。先前迭代算法中使用的局部优化过程被证明是该多面体的分离预言机。由于这些过程通常具有多项式时间复杂度，应用椭球方法可立即为这些问题提供多项式时间算法。