Offline model-based optimization aims to maximize a black-box objective function with a static dataset of designs and their scores. In this paper, we focus on biological sequence design to maximize some sequence score. A recent approach employs bidirectional learning, combining a forward mapping for exploitation and a backward mapping for constraint, and it relies on the neural tangent kernel (NTK) of an infinitely wide network to build a proxy model. Though effective, the NTK cannot learn features because of its parametrization, and its use prevents the incorporation of powerful pre-trained Language Models (LMs) that can capture the rich biophysical information in millions of biological sequences. We adopt an alternative proxy model, adding a linear head to a pre-trained LM, and propose a linearization scheme. This yields a closed-form loss and also takes into account the biophysical information in the pre-trained LM. In addition, the forward mapping and the backward mapping play different roles and thus deserve different weights during sequence optimization. To achieve this, we train an auxiliary model and leverage its weak supervision signal via a bi-level optimization framework to effectively learn how to balance the two mappings. Further, by extending the framework, we develop the first learning rate adaptation module \textit{Adaptive}-$\eta$, which is compatible with all gradient-based algorithms for offline model-based optimization. Experimental results on DNA/protein sequence design tasks verify the effectiveness of our algorithm. Our code is available~\href{https://anonymous.4open.science/r/BIB-ICLR2023-Submission/README.md}{here.}

翻译：离线模型优化旨在利用静态数据集（包含设计方案及其分数）最大化黑箱目标函数。本文聚焦于生物序列设计，以最大化序列分数。近期方法采用双向学习，结合正向映射（利用已有知识）和反向映射（施加约束），并依赖无限宽网络的神经正切核（NTK）构建代理模型。尽管有效，NTK因其参数化方式无法学习特征，且其使用阻碍了集成能够捕获数百万生物序列中丰富生物物理信息的强大预训练语言模型（LM）。我们采用替代代理模型，在预训练LM上添加线性头部，并提出线性化方案。这产生了闭合形式的损失函数，同时利用了预训练LM中的生物物理信息。此外，正向映射和反向映射发挥不同作用，因此在序列优化过程中应赋予不同权重。为实现此目标，我们训练辅助模型，并通过双层优化框架利用其弱监督信号，有效学习如何平衡两种映射。进一步，通过扩展该框架，我们开发了首个学习率自适应模块 \textit{Adaptive}-$\eta$，该模块兼容所有基于梯度的离线模型优化算法。在DNA/蛋白质序列设计任务上的实验结果验证了算法的有效性。我们的代码在此处提供。