We study offline black-box optimization (BBO), aiming to discover improved designs from an offline dataset of designs and labels, a problem common in robotics and DNA with limited labeled samples. While recent work applies autoregressive LLMs to BBO by formatting tasks as natural-language prompts, their left-to-right design generation struggles to capture the strong bidirectional dependencies inherent in design problems. To address this, we propose adapting diffusion LLMs to offline BBO to leverage their bidirectional modeling capabilities. However, a domain gap exists between the natural text pre-training of diffusion LLMs and the heterogeneous signals in BBO (prompts, designs, and labels). To bridge this gap, we construct a unified prompt--response corpus and introduce delimiter tokens to explicitly mark field boundaries for domain adaptation. We further propose a two-stage post-training framework to align the diffusion LLM generation with high-label designs. The first stage performs supervised fine-tuning on the unified dataset via masked-response prediction, and the second stage adopts reinforcement learning with rewards defined by label improvements. Our method achieves state-of-the-art results on Design-Bench under small-data settings with highly efficient training, requiring only $1.5$ H100 GPU hours for discrete tasks. Code for our work is available here: https://github.com/zpointS/DiBO.

翻译：我们研究离线黑盒优化（BBO），旨在从包含设计与标签的离线数据集中发现改进的设计方案，这在机器人学和DNA领域标注样本有限的问题中尤为常见。虽然近期工作通过将任务格式化为自然语言提示，将自回归大型语言模型应用于BBO，但其从左到右的设计生成过程难以捕捉设计问题中固有的强双向依赖关系。为解决此问题，我们提出将扩散语言模型适配至离线BBO，以利用其双向建模能力。然而，扩散语言模型在自然文本预训练与BBO中的异质信号（提示、设计与标签）之间存在领域差异。为弥合这一差距，我们构建了一个统一的提示-响应语料库，并引入分隔符标记以明确标注字段边界进行领域适配。我们进一步提出两阶段后训练框架，使扩散语言模型的生成与高标签设计对齐。第一阶段通过掩码响应预测在统一数据集上进行监督微调，第二阶段采用以标签改进为奖励的强化学习。我们的方法在Design-Bench小样本设置下取得了最先进的结果，且训练效率极高，离散任务仅需1.5个H100 GPU小时。代码链接：https://github.com/zpointS/DiBO。