For residual learning that refines existing behavior, sample efficiency depends on two things: how much information each rollout returns, and how efficiently the learner uses that information. Reinforcement learning's standard scalar reward carries far less information than the directional task error that defines the task. Random exploration further discards whatever information each rollout returns. Through residual learning with directional task-error supervision and a task error model that drives sample selection, we achieve stable three-, four-, and five-ball juggling on anthropomorphic Barrett WAM arms. Despite planning and controlling through a simple, idealized stack, the system converges from the second attempt. The first attempt drops, after which task error decreases monotonically without further failures. In comparison, five-ball juggling typically takes humans years of practice. We compare residual learners across two ternary axes, the directional information in the learning feedback and the commitment of the analytic prior, spanning Newton-style Jacobian updates, Composite Bayesian Optimization, and stochastic search methods. Both axes prove necessary: neither directional feedback nor an informative prior suffices alone, and the simplest method that combines them, a fixed-Jacobian Newton update, is the most reliable. The learned residual tolerates substantial prior misalignment and degraded joint tracking, affecting mainly convergence speed. The bottleneck for residual learning on real robots is therefore the information content of the supervision signal and how the learner uses it, not the accuracy of the surrounding stack. Video documentation of all experiments is available at https://kai-ploeger.com/residual-juggling.

翻译：对于优化现有行为的残差学习，样本效率取决于两个因素：每次 rollout 返回的信息量，以及学习器利用这些信息的效率。强化学习的标准标量奖励携带的信息远少于定义任务的方向性任务误差。随机探索进一步丢弃了每次 rollout 返回的信息。通过结合方向性任务误差监督的残差学习以及驱动样本选择的任务误差模型，我们在拟人化的 Barrett WAM 机械臂上实现了稳定三球、四球和五球杂耍。尽管规划和控制基于简单、理想化的堆栈，系统从第二次尝试起便收敛。第一次尝试失败后，任务误差单调递减，且未进一步出现失误。相比之下，五球杂耍通常需要人类数年的练习。我们沿两个三元轴比较了残差学习器：学习反馈中的方向性信息，以及解析先验的承诺程度，涵盖牛顿式雅可比更新、复合贝叶斯优化和随机搜索方法。两个轴均被证明是必要的：单独的方向性反馈或信息性先验都不足以胜任，而结合两者的最简方法——固定雅可比牛顿更新——最为可靠。学习到的残差能容忍显著的先验失准和退化的关节跟踪，主要影响收敛速度。因此，真实机器人上残差学习的瓶颈在于监督信号的信息内容及其利用方式，而非周围堆栈的精度。所有实验的视频记录可在 https://kai-ploeger.com/residual-juggling 获取。