We present task-oriented Koopman-based control that utilizes end-to-end reinforcement learning and contrastive encoder to simultaneously learn the Koopman latent embedding, operator, and associated linear controller within an iterative loop. By prioritizing the task cost as the main objective for controller learning, we reduce the reliance of controller design on a well-identified model, which, for the first time to the best of our knowledge, extends Koopman control from low to high-dimensional, complex nonlinear systems, including pixel-based tasks and a real robot with lidar observations. Code and videos are available \href{https://sites.google.com/view/kpmlilatsupp/}{here}.

翻译：我们提出了一种面向任务的基于Koopman的控制方法，该方法利用端到端强化学习和对比编码器，在迭代循环中同时学习Koopman潜在嵌入、算子及相应的线性控制器。通过将任务代价作为控制器学习的主要目标，我们降低了对良好辨识模型的依赖——据我们所知，这是首次将Koopman控制从低维系统推广至高维复杂非线性系统，包括基于像素的任务和配备激光雷达观测的真实机器人。代码与视频可在此处获取：\href{https://sites.google.com/view/kpmlilatsupp/}{链接}。