Reinforcement learning (RL) has become a powerful paradigm for robot learning, particularly in sim-to-real settings, but its broader adoption remains limited by the engineering pipeline surrounding the algorithms. Building tasks, shaping rewards, and tuning hyperparameters require substantial expert effort, making RL workflows costly and difficult to scale. We introduce HARBOR, an agentic framework that frames robot RL automation as a harness-engineering problem: given a simulator codebase and a task specification, it automates the workflow from environment setup to policy training in simulation. HARBOR decomposes such high-level objectives into bounded stages executed by specialized agents through standardized commands, persistent artifacts, executable gates, and reusable knowledge, and scales iteration via decentralized parallel trials and experience learning across runs. We evaluate HARBOR across 6 benchmarks and 16 tasks in total, spanning manipulation, locomotion, and bimanual dexterous control. We demonstrate that HARBOR automates the simulation RL workflow end-to-end, designs rewards, tunes algorithms to match or improve over default configurations, and reduces engineering effort at practical token and wall-clock cost; the resulting policies can also be transferred to real robots.

翻译：摘要：强化学习已成为机器人学习领域的重要范式，尤其在仿真到现实迁移场景中成效显著。然而，其算法周边工程流水线的复杂性严重制约了该技术的广泛采纳：任务构建、奖励塑造与超参数调优均需大量专家投入，导致RL工作流成本高昂且难以规模化。我们提出HARBOR框架——一种将机器人RL自动化问题重构为流程工程问题的自主化框架：给定仿真代码库与任务规范，该框架能实现从环境搭建到仿真策略训练的全流程自动化。HARBOR将高层目标解耦为边界分明的阶段，由专用智能体通过标准化指令、持久化工件、可执行网关与可复用知识进行协同执行；并借助去中心化并行试验与跨次经验学习实现迭代扩展。我们在涵盖操作、移动与双臂灵巧控制的6个基准测试与16项任务中完成了系统评估。实验表明，HARBOR可实现仿真RL工作流端到端自动化，自动设计奖励函数并调优算法以匹配或超越默认配置性能，同时以实际的令牌开销与时钟成本显著降低工程投入，最终生成的策略亦可成功迁移至真实机器人。