Balancing competing objectives is omnipresent across disciplines, from drug design to autonomous systems. Multi-objective Bayesian optimization is a promising solution for such expensive, black-box problems: it fits probabilistic surrogates and selects new designs via an acquisition function that balances exploration and exploitation. In practice, it requires tailored choices of surrogate and acquisition that rarely transfer to the next problem, is myopic when multi-step planning is often required, and adds refitting overhead, particularly in parallel or time-sensitive loops. We present TAMO, a fully amortized, universal policy for multi-objective black-box optimization. TAMO uses a transformer architecture that operates across varying input and objective dimensions, enabling pretraining on diverse corpora and transfer to new problems without retraining: at test time, the pretrained model proposes the next design with a single forward pass. We pretrain the policy with reinforcement learning to maximize cumulative hypervolume improvement over full trajectories, conditioning on the entire query history to approximate the Pareto frontier. Across synthetic benchmarks and real tasks, TAMO produces fast proposals, reducing proposal time by 50-1000x versus alternatives while matching or improving Pareto quality under tight evaluation budgets. These results show that transformers can perform multi-objective optimization entirely in-context, eliminating per-task surrogate fitting and acquisition engineering, and open a path to foundation-style, plug-and-play optimizers for scientific discovery workflows.

翻译：平衡相互冲突的目标是跨学科领域的普遍挑战，从药物设计到自主系统均不例外。多目标贝叶斯优化为解决这类昂贵的黑箱问题提供了有效方案：它通过拟合概率代理模型，并利用能够平衡探索与利用的采集函数选择新设计。然而在实际应用中，该方法需要针对特定问题定制代理模型与采集函数，难以迁移至新问题；在多步规划需求下存在短视缺陷；且并行或时间敏感循环中需不断重构模型，增加了计算开销。本文提出TAMO——一种完全摊销化的通用多目标黑箱优化策略。TAMO采用Transformer架构，可处理不同输入与目标维度的优化问题，通过多样化语料库的预训练实现无需重新训练即可迁移至新问题的能力：测试阶段，预训练模型通过单次前向传播即可生成下一组设计方案。我们利用强化学习对策略进行预训练，以最大化完整轨迹的累积超体积改进为目标，基于历史查询序列近似帕累托前沿。在合成基准测试与真实任务中，TAMO展现出高效提案能力：在严格评估预算下，其提案时间较现有方法减少50-1000倍，同时保持或提升帕累托质量。这些结果表明，Transformer能够完全基于上下文实现多目标优化，省去了逐任务代理模型拟合与采集函数设计的繁琐流程，为科学发现流程中构建即插即用型基础优化器开辟了新路径。