Many communication and control problems are cast as multi-objective Markov decision processes (MOMDPs). The complete solution to an MOMDP is the Pareto front. Much of the literature approximates this front via scalarization into single-objective MDPs. Recent work has begun to characterize the full front in discounted or simple bi-objective settings by exploiting its geometry. In this work, we characterize the exact front in average-cost MOMDPs. We show that the front is a continuous, piecewise-linear surface lying on the boundary of a convex polytope. Each vertex corresponds to a deterministic policy, and adjacent vertices differ in exactly one state. Each edge is realized as a convex combination of the policies at its endpoints, with the mixing coefficient given in closed form. We apply these results to a remote state estimation problem, where each vertex on the front corresponds to a threshold policy. The exact Pareto front and solutions to certain non-convex MDPs can be obtained without explicitly solving any MDP.

翻译：许多通信与控制问题可归结为多目标马尔可夫决策过程（MOMDPs）。MOMDP的完整解即为帕累托前沿。现有文献多通过标量化方法将其转化为单目标MDP进行近似求解。近期研究开始利用前沿几何特性，在折扣或简单双目标场景中刻画完整前沿。本文针对平均代价MOMDP，首次刻画其精确前沿。研究表明：该前沿为位于凸多面体边界上的连续分段线性曲面，每个顶点对应一个确定性策略，相邻顶点仅在一个状态上存在差异；每条边可通过端点策略的凸组合实现，且混合系数具有闭式解。我们将这些结论应用于远程状态估计问题，其中前沿上的每个顶点对应一个阈值策略。在不显式求解任何MDP的情况下，即可获得精确帕累托前沿及某些非凸MDP的解。