Decision Transformers (DT) have demonstrated strong performances in offline reinforcement learning settings, but quickly adapting to unseen novel tasks remains challenging. To address this challenge, we propose a new framework, called Hyper-Decision Transformer (HDT), that can generalize to novel tasks from a handful of demonstrations in a data- and parameter-efficient manner. To achieve such a goal, we propose to augment the base DT with an adaptation module, whose parameters are initialized by a hyper-network. When encountering unseen tasks, the hyper-network takes a handful of demonstrations as inputs and initializes the adaptation module accordingly. This initialization enables HDT to efficiently adapt to novel tasks by only fine-tuning the adaptation module. We validate HDT's generalization capability on object manipulation tasks. We find that with a single expert demonstration and fine-tuning only 0.5% of DT parameters, HDT adapts faster to unseen tasks than fine-tuning the whole DT model. Finally, we explore a more challenging setting where expert actions are not available, and we show that HDT outperforms state-of-the-art baselines in terms of task success rates by a large margin.

翻译：决策Transformer（DT）在离线强化学习场景中展现出强大性能，但快速适应未知新任务仍具挑战性。为此，我们提出名为超决策Transformer（HDT）的新框架，该框架能够以数据高效和参数高效的方式，通过少量示范实现对新任务的泛化。为实现这一目标，我们提出在基础DT中增加一个自适应模块，其参数由超网络初始化。当遇到未知任务时，超网络将少量示范作为输入，并据此初始化自适应模块。这种初始化机制使得HDT仅通过微调自适应模块即可高效适应新任务。我们在物体操控任务上验证了HDT的泛化能力。研究发现，仅凭单个专家示范并微调DT参数总量的0.5%，HDT适应未知任务的速度就优于微调整个DT模型。最后，我们探索了专家动作不可用这一更具挑战性的场景，实验表明HDT在任务成功率上大幅超越现有最优基线模型。