We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training. Code is available at: https://pei-xu.github.io/guitar.

翻译：本文提出了一种新颖的方法，用于在需要高时序精度的双手协调控制任务中合成物理模拟手的灵巧运动。我们的方法并非直接学习控制双手的联合策略，而是通过协作学习实现双手控制，将每只手视为独立智能体。首先分别训练每只手的独立策略，随后在集中式环境中通过潜在空间操作进行同步，以形成双手控制的联合策略。通过这种方式，我们避免了在更高维度的双手联合状态-动作空间中直接进行策略学习，从而显著提升了整体训练效率。我们在具有挑战性的吉他演奏任务中验证了所提方法的有效性。通过本方法训练的虚拟吉他手能够根据非结构化的通用吉他练习动作参考数据合成运动，并依据输入的不存在于参考数据中的吉他谱，精准演奏具有复杂和弦按压与拨弦模式的多样化节奏。随本文附上我们采集的用于策略训练参考的运动捕捉数据。代码发布于：https://pei-xu.github.io/guitar。