Despite impressive dexterous manipulation capabilities enabled by learning-based approaches, we are yet to witness widespread adoption beyond well-resourced laboratories. This is likely due to practical limitations, such as significant computational burden, inscrutable learned behaviors, sensitivity to initialization, and the considerable technical expertise required for implementation. In this work, we investigate the utility of Koopman operator theory in alleviating these limitations. Koopman operators are simple yet powerful control-theoretic structures to represent complex nonlinear dynamics as linear systems in higher dimensions. Motivated by the fact that complex nonlinear dynamics underlie dexterous manipulation, we develop a Koopman operator-based imitation learning framework to learn the desired motions of both the robotic hand and the object simultaneously. We show that Koopman operators are surprisingly effective for dexterous manipulation and offer a number of unique benefits. Notably, policies can be learned analytically, drastically reducing computation burden and eliminating sensitivity to initialization and the need for painstaking hyperparameter optimization. Our experiments reveal that a Koopman operator-based approach can perform comparably to state-of-the-art imitation learning algorithms in terms of success rate and sample efficiency, while being an order of magnitude faster.

翻译：尽管基于学习的方法赋予了令人瞩目的灵巧操作能力，但此类技术仍未能广泛普及于资源充足的实验室之外。这很可能源于实际限制，例如显著的计算负担、难以解读的学习行为、对初始化的敏感度，以及实施所需的大量专业知识。在本研究中，我们探讨了库普曼算子理论在缓解这些限制方面的效用。库普曼算子是一种简洁而强大的控制理论结构，能够将复杂的非线性动力学表征为高维空间中的线性系统。基于灵巧操作底层存在的复杂非线性动力学这一事实，我们开发了一种基于库普曼算子的模仿学习框架，以同时学习机器人手部与物体的期望运动。研究表明，库普曼算子在灵巧操作中具有惊人的有效性，并提供了诸多独特优势。值得注意的是，策略可通过解析方式学习，大幅降低计算负担，消除对初始化的敏感性以及对繁琐超参数优化的需求。我们的实验表明，基于库普曼算子的方法在成功率和样本效率上可与最先进的模仿学习算法相媲美，同时速度快出一个数量级。