Diffusion models have demonstrated highly-expressive generative capabilities in vision and NLP. Recent studies in reinforcement learning (RL) have shown that diffusion models are also powerful in modeling complex policies or trajectories in offline datasets. However, these works have been limited to single-task settings where a generalist agent capable of addressing multi-task predicaments is absent. In this paper, we aim to investigate the effectiveness of a single diffusion model in modeling large-scale multi-task offline data, which can be challenging due to diverse and multimodal data distribution. Specifically, we propose Multi-Task Diffusion Model (\textsc{MTDiff}), a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis in multi-task offline settings. \textsc{MTDiff} leverages vast amounts of knowledge available in multi-task data and performs implicit knowledge sharing among tasks. For generative planning, we find \textsc{MTDiff} outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D. For data synthesis, \textsc{MTDiff} generates high-quality data for testing tasks given a single demonstration as a prompt, which enhances the low-quality datasets for even unseen tasks.

翻译：扩散模型在视觉和自然语言处理领域展现了强大的生成能力。近期强化学习研究表明，扩散模型在建模离线数据集中的复杂策略或轨迹方面同样表现卓越。然而，现有工作仅限于单一任务场景，未涉及能够处理多任务困境的通用型智能体。本文旨在探究单一扩散模型在建模大规模多任务离线数据时的有效性——此类数据因分布多样且多模态而极具挑战性。具体而言，我们提出多任务扩散模型（\textsc{MTDiff}），这是一种基于扩散的方法，融合了Transformer骨干网络与提示学习，用于多任务离线场景下的生成式规划与数据合成。\textsc{MTDiff}可充分利用多任务数据中的海量知识，并实现任务间的隐式知识共享。在生成式规划方面，我们发现\textsc{MTDiff}在Meta-World的50个任务和Maze2D的8张地图上均优于现有最优算法。在数据合成方面，\textsc{MTDiff}能以单条示范作为提示生成高质量测试任务数据，从而提升低质量数据集的质量，甚至可为未见任务生成有效数据。