This paper investigates how representation learning can enable optimal control in unknown and complex dynamics, such as chaotic and non-linear systems, without relying on prior domain knowledge of the dynamics. The core idea is to establish an equivariant geometry that is diffeomorphic to the manifold defined by a dynamical system and to perform optimal control within this corresponding geometry, which is a non-trivial task. To address this challenge, Koopman Embed to Equivariant Control (KEEC) is introduced for model learning and control. Inspired by Lie theory, KEEC begins by learning a non-linear dynamical system defined on a manifold and embedding trajectories into a Lie group. Subsequently, KEEC formulates an equivariant value function equation in reinforcement learning on the equivariant geometry, ensuring an invariant effect as the value function on the original manifold. By deriving analytical-form optimal actions on the equivariant value function, KEEC theoretically achieves quadratic convergence for the optimal equivariant value function by leveraging the differential information on the equivariant geometry. The effectiveness of KEEC is demonstrated in challenging dynamical systems, including chaotic ones like Lorenz-63. Notably, our findings indicate that isometric and isomorphic loss functions, ensuring the compactness and smoothness of geometry, outperform loss functions without these properties.

翻译：本文研究了表示学习如何能够在未知且复杂的动力学（如混沌和非线性系统）中实现最优控制，且无需依赖对动力学的先验领域知识。核心思想是建立一个与动力系统定义的流形微分同胚的等变几何，并在该对应几何中进行最优控制，这是一项非平凡的任务。为应对这一挑战，本文引入了Koopman嵌入到等变控制（KEEC）方法进行模型学习与控制。受李群理论启发，KEEC首先学习一个定义在流形上的非线性动力系统，并将轨迹嵌入到李群中。随后，KEEC在等变几何上构建了强化学习中的等价值函数方程，确保原始流形上的值函数具有不变效应。通过推导等价值函数上的解析形式最优动作，KEEC在理论上利用等变几何上的微分信息实现了最优等价值函数的二次收敛。KEEC的有效性在具有挑战性的动力系统中得到验证，包括如Lorenz-63等混沌系统。值得注意的是，我们的研究结果表明，保证几何紧致性与光滑性的等距同构损失函数优于不具备这些性质的损失函数。