Combinatorial assembly uses standardized unit primitives to build objects that satisfy user specifications. This paper studies assembly sequence planning (ASP) for physical combinatorial assembly. Given the shape of the desired object, the goal is to find a sequence of actions for placing unit primitives to build the target object. In particular, we aim to ensure the planned assembly sequence is physically executable. However, ASP for combinatorial assembly is particularly challenging due to its combinatorial nature. To address the challenge, we employ deep reinforcement learning to learn a construction policy for placing unit primitives sequentially to build the desired object. Specifically, we design an online physics-aware action mask that filters out invalid actions, which effectively guides policy learning and ensures violation-free deployment. In the end, we apply the proposed method to Lego assembly with more than 250 3D structures. The experiment results demonstrate that the proposed method plans physically valid assembly sequences to build all structures, achieving a $100\%$ success rate, whereas the best comparable baseline fails more than $40$ structures. Our implementation is available at \url{https://github.com/intelligent-control-lab/PhysicsAwareCombinatorialASP}.

翻译：组合装配利用标准化单元基元构建满足用户规格的物体。本文研究物理组合装配中的装配序列规划问题。给定目标物体的形状，规划目标是为放置单元基元以构建目标物体寻找动作序列。我们特别关注确保所规划的装配序列在物理上可执行。然而，组合装配的ASP因其组合性质而极具挑战性。为应对这一挑战，我们采用深度强化学习来学习顺序放置单元基元以构建目标物体的建造策略。具体而言，我们设计了一种在线物理感知动作掩码，用于过滤无效动作，从而有效指导策略学习并确保无违规部署。最终，我们将所提方法应用于包含超过250种三维结构的乐高装配任务。实验结果表明，所提方法能为所有结构规划物理有效的装配序列，达成$100\%$的成功率，而最佳可比基线方法在超过$40$个结构上失败。我们的实现代码已发布于\url{https://github.com/intelligent-control-lab/PhysicsAwareCombinatorialASP}。