Despite impressive dexterous manipulation capabilities enabled by learning-based approaches, we are yet to witness widespread adoption beyond well-resourced laboratories. This is likely due to practical limitations, such as significant computational burden, inscrutable learned behaviors, sensitivity to initialization, and the considerable technical expertise required for implementation. In this work, we investigate the utility of Koopman operator theory in alleviating these limitations. Koopman operators are simple yet powerful control-theoretic structures to represent complex nonlinear dynamics as linear systems in higher dimensions. Motivated by the fact that complex nonlinear dynamics underlie dexterous manipulation, we develop a Koopman operator-based imitation learning framework to learn the desired motions of both the robotic hand and the object simultaneously. We show that Koopman operators are surprisingly effective for dexterous manipulation and offer a number of unique benefits. Notably, policies can be learned analytically, drastically reducing computation burden and eliminating sensitivity to initialization and the need for painstaking hyperparameter optimization. Our experiments reveal that a Koopman operator-based approach can perform comparably to state-of-the-art imitation learning algorithms in terms of success rate and sample efficiency, while being an order of magnitude faster.

翻译：尽管基于学习方法已展现出令人瞩目的灵巧操作能力，但其在资源充足的实验室之外仍未得到广泛采用。这很可能源于实际应用限制，例如显著的计算负担、难以解释的学习行为、对初始化的敏感性，以及实施所需的大量专业技术知识。在本研究中，我们探究了库普曼算子理论在缓解上述限制方面的效用。库普曼算子是一类简洁而强大的控制理论结构，能够将复杂非线性动力学表征为高维线性系统。基于灵巧操作背后蕴含复杂非线性动力学这一事实，我们开发了一种基于库普曼算子的模仿学习框架，可同时学习机械手与目标物体的期望运动。研究表明，库普曼算子对灵巧操作具有惊人的有效性，并提供多项独特优势。具体而言，策略可通过解析方式学习，从而大幅降低计算负担，消除对初始化的敏感性以及对繁琐超参数优化的需求。实验结果表明，基于库普曼算子的方法在成功率和样本效率方面可与最先进的模仿学习算法相媲美，同时速度提升一个数量级。