This paper proposes a comprehensive hierarchical control framework for autonomous decision-making arising in robotics and autonomous systems. In a typical hierarchical control architecture, high-level decision making is often characterised by discrete state and decision/control sets. However, a rational decision is usually affected by not only the discrete states of the autonomous system, but also the underlying continuous dynamics even the evolution of its operational environment. This paper proposes a holistic and comprehensive design process and framework for this type of challenging problems, from new modelling and design problem formulation to control design and stability analysis. It addresses the intricate interplay between traditional continuous systems dynamics utilized at the low levels for control design and discrete Markov decision processes (MDP) for facilitating high-level decision making. We model the decision making system in complex environments as a hybrid system consisting of a controlled MDP and autonomous (i.e. uncontrolled) continuous dynamics. Consequently, the new formulation is called as hybrid Markov decision process (HMDP). The design problem is formulated with a focus on ensuring both safety and optimality while taking into account the influence of both the discrete and continuous state variables of different levels. With the help of the model predictive control (MPC) concept, a decision maker design scheme is proposed for the proposed hybrid decision making model. By carefully designing key ingredients involved in this scheme, it is shown that the recursive feasibility and stability of the proposed autonomous decision making scheme are guaranteed. The proposed framework is applied to develop an autonomous lane changing system for intelligent vehicles.

翻译：本文提出了一种面向机器人与自主系统决策问题的综合分层控制框架。在典型的分层控制架构中，高层决策通常由离散状态及决策/控制集合表征。然而，理性决策不仅受自主系统离散状态的影响，还受底层连续动态特性乃至运行环境演化的制约。针对此类挑战性问题，本文提出了一套从新型建模与设计问题形式化到控制设计与稳定性分析的整体化综合设计流程与框架，揭示了用于底层控制设计的传统连续系统动态特性与促进高层决策的离散马尔可夫决策过程（MDP）之间复杂的交互机制。我们将复杂环境下的决策系统建模为由受控MDP与自主（即非受控）连续动态特性组成的混合系统，据此将新形式化模型称为混合马尔可夫决策过程（HMDP）。该设计问题在兼顾安全性与最优性的同时，重点考虑了不同层级离散与连续状态变量的影响。借助模型预测控制（MPC）概念，为所提出的混合决策模型设计了决策制定方案。通过精心设计该方案的关键要素，证明了所提出的自主决策方案的递归可行性与稳定性。最后，将该框架应用于智能车辆自主换道系统的开发。