End-to-end Transformers have demonstrated an impressive success rate for Embodied Instruction Following when the environment has been seen in training. However, they tend to struggle when deployed in an unseen environment. This lack of generalizability is due to the agent's insensitivity to subtle changes in natural language instructions. To mitigate this issue, we propose explicitly aligning the agent's hidden states with the instructions via contrastive learning. Nevertheless, the semantic gap between high-level language instructions and the agent's low-level action space remains an obstacle. Therefore, we further introduce a novel concept of meta-actions to bridge the gap. Meta-actions are ubiquitous action patterns that can be parsed from the original action sequence. These patterns represent higher-level semantics that are intuitively aligned closer to the instructions. When meta-actions are applied as additional training signals, the agent generalizes better to unseen environments. Compared to a strong multi-modal Transformer baseline, we achieve a significant 4.5% absolute gain in success rate in unseen environments of ALFRED Embodied Instruction Following. Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents. The code is available at: https://github.com/joeyy5588/LACMA.

翻译：端到端Transformer在训练环境中已展现出令人印象深刻的具身指令跟随成功率，但在未见过环境中部署时往往表现不佳。这种泛化能力的缺失源于智能体对自然语言指令中细微变化的不敏感性。为缓解该问题，我们提出通过对比学习显式对齐智能体隐藏状态与指令。然而，高层语言指令与智能体低层动作空间之间的语义鸿沟仍是障碍。为此，我们进一步引入元动作这一全新概念以弥合鸿沟。元动作是从原始动作序列中解析出的泛化动作模式，这些模式代表与指令直觉上更接近的高层语义。当元动作作为附加训练信号使用时，智能体在未见环境中展现出更强的泛化能力。相比强大的多模态Transformer基线，我们在ALFRED具身指令跟随的未见环境中实现了4.5%的绝对成功率提升。进一步分析表明，对比学习目标与元动作在实现最佳效果时具有互补性，且最终智能体能更好地将其状态与对应指令对齐，使其更适用于真实世界的具身智能体。代码开源地址：https://github.com/joeyy5588/LACMA。