Flexible manufacturing requires robot systems that can adapt to constantly changing tasks, objects, and environments. However, traditional robot programming is labor-intensive and inflexible, while existing learning-based assembly methods often suffer from weak positional generalization, complex multi-stage designs, and limited multi-skill integration capability. To address these issues, this paper proposes ATG-MoE, an end-to-end autoregressive trajectory generation method with mixture of experts for assembly skill learning from demonstration. The proposed method establishes a closed-loop mapping from multi-modal inputs, including RGB-D observations, natural language instructions, and robot proprioception to manipulation trajectories. It integrates multi-modal feature fusion for scene and task understanding, autoregressive sequence modeling for temporally coherent trajectory generation, and a mixture-of-experts architecture for unified multi-skill learning. In contrast to conventional methods that separate visual perception and control or train different skills independently, ATG-MoE directly incorporates visual information into trajectory generation and supports efficient multi-skill integration within a single model. We train and evaluate the proposed method on eight representative assembly skills from a pressure-reducing valve assembly task. Experimental results show that ATG-MoE achieves strong overall performance in simulation, with an average grasp success rate of 96.3% and an average overall success rate of 91.8%, while also demonstrating strong generalization and effective multi-skill integration. Real-world experiments further verify its practicality for multi-skill industrial assembly. The project page can be found at https://hwh23.github.io/ATG-MoE

翻译：摘要：柔性制造要求机器人系统能够适应不断变化的任务、物体与环境。然而，传统的机器人编程方式劳动密集且缺乏灵活性，而现有的基于学习的装配方法常面临位置泛化能力弱、多阶段设计复杂以及多技能集成能力有限等问题。针对这些挑战，本文提出ATG-MoE——一种端到端的基于混合专家机制的自回归轨迹生成方法，用于从示教中学习装配技能。该方法建立了从多模态输入（包括RGB-D观测、自然语言指令与机器人本体感知）到操作轨迹的闭环映射，融合了场景与任务理解所需的多模态特征融合、时序连贯轨迹生成所需的自回归序列建模，以及统一多技能学习所需的混合专家架构。与将视觉感知与控制分离或独立训练不同技能的传统方法不同，ATG-MoE直接将视觉信息融入轨迹生成过程，并在单一模型内支持高效的多技能集成。我们在减压阀装配任务的八种代表性装配技能上对所提方法进行了训练与评估。实验结果表明，ATG-MoE在仿真环境中取得了优异的整体性能，平均抓取成功率达96.3%，平均整体成功率达91.8%，同时展现出强大的泛化能力与有效的多技能集成能力。真实场景实验进一步验证了其在多技能工业装配中的实用性。项目页面详见：https://hwh23.github.io/ATG-MoE