Learning how to effectively control unknown dynamical systems is crucial for intelligent autonomous systems. This task becomes a significant challenge when the underlying dynamics are changing with time. Motivated by this challenge, this paper considers the problem of controlling an unknown Markov jump linear system (MJS) to optimize a quadratic objective. By taking a model-based perspective, we consider identification-based adaptive control of MJSs. We first provide a system identification algorithm for MJS to learn the dynamics in each mode as well as the Markov transition matrix, underlying the evolution of the mode switches, from a single trajectory of the system states, inputs, and modes. Through martingale-based arguments, sample complexity of this algorithm is shown to be $\mathcal{O}(1/\sqrt{T})$. We then propose an adaptive control scheme that performs system identification together with certainty equivalent control to adapt the controllers in an episodic fashion. Combining our sample complexity results with recent perturbation results for certainty equivalent control, we prove that when the episode lengths are appropriately chosen, the proposed adaptive control scheme achieves $\mathcal{O}(\sqrt{T})$ regret, which can be improved to $\mathcal{O}(polylog(T))$ with partial knowledge of the system. Our proof strategy introduces innovations to handle Markovian jumps and a weaker notion of stability common in MJSs. Our analysis provides insights into system theoretic quantities that affect learning accuracy and control performance. Numerical simulations are presented to further reinforce these insights.

翻译：学习如何有效控制未知动态系统对于智能自主系统至关重要。当底层动态特性随时间变化时，该任务成为一项重大挑战。受此挑战驱动，本文研究了控制未知马尔可夫跳跃线性系统以优化二次目标函数的问题。基于模型化视角，我们探讨了MJS的基于辨识的自适应控制方法。首先提出了一种MJS系统辨识算法，可从系统状态、输入和模态的单条轨迹中学习各模态的动态特性以及表征模态切换演化规律的马尔可夫转移矩阵。通过基于鞅的论证，证明该算法的样本复杂度为$\mathcal{O}(1/\sqrt{T})$。随后提出一种自适应控制方案，将系统辨识与确定性等价控制相结合，以分段方式自适应调整控制器。将样本复杂度结果与近期确定性等价控制的扰动结果相结合，证明当分段长度选择适当时，所提自适应控制方案可实现$\mathcal{O}(\sqrt{T})$遗憾界，若具备系统部分先验知识可进一步提升至$\mathcal{O}(polylog(T))$。我们的证明策略引入了创新方法以处理马尔可夫跳跃现象及MJS中常见的弱稳定性概念。理论分析揭示了影响学习精度与控制性能的系统理论量。数值仿真进一步验证了这些理论见解。