Advancements in AI for science unlocks capabilities for critical drug discovery tasks such as protein-ligand binding affinity prediction. However, current models overfit to existing oversimplified datasets that does not represent naturally occurring and biologically relevant proteins with modifications. In this work, we curate a complete and modification-aware version of the widely used DAVIS dataset by incorporating 4,032 kinase-ligand pairs involving substitutions, insertions, deletions, and phosphorylation events. This enriched dataset enables benchmarking of predictive models under biologically realistic conditions. Based on this new dataset, we propose three benchmark settings-Augmented Dataset Prediction, Wild-Type to Modification Generalization, and Few-Shot Modification Generalization-designed to assess model robustness in the presence of protein modifications. Through extensive evaluation of both docking-free and docking-based methods, we find that docking-based model generalize better in zero-shot settings. In contrast, docking-free models tend to overfit to wild-type proteins and struggle with unseen modifications but show notable improvement when fine-tuned on a small set of modified examples. We anticipate that the curated dataset and benchmarks offer a valuable foundation for developing models that better generalize to protein modifications, ultimately advancing precision medicine in drug discovery. The benchmark is available at: https://github.com/ZhiGroup/DAVIS-complete

翻译：人工智能在科学领域的进展为关键药物发现任务（如蛋白质-配体结合亲和力预测）解锁了新能力。然而，当前模型过度拟合于现有的过度简化数据集，这些数据集未能代表自然界中存在且具有生物学相关性的修饰蛋白质。本研究通过整合涉及替换、插入、缺失和磷酸化事件的4,032个激酶-配体对，构建了一个完整且修饰感知的广泛使用的DAVIS数据集版本。这一增强后的数据集能够在生物学现实条件下对预测模型进行基准测试。基于此新数据集，我们提出了三个基准设置——增强数据集预测、野生型到修饰的泛化以及少样本修饰泛化——旨在评估模型在蛋白质修饰存在下的鲁棒性。通过对无对接方法和基于对接方法的广泛评估，我们发现基于对接的模型在零样本设置中表现出更好的泛化能力。相比之下，无对接模型倾向于过度拟合野生型蛋白质，在未见过的修饰上表现不佳，但在少量修饰示例上进行微调后显示出显著改进。我们预期，这一精心构建的数据集和基准将为开发能更好泛化至蛋白质修饰的模型提供宝贵基础，最终推动药物发现中的精准医学发展。基准测试可通过以下链接获取：https://github.com/ZhiGroup/DAVIS-complete