Published transfer-BO comparisons often estimate an average treatment effect of acquisition choice over hidden regime variables, while practitioners need the conditional effect for their specific prior quality, budget ratio, and metric. An audit of 40 transfer-BO papers from NeurIPS, ICML, ICLR, AISTATS, UAI, TMLR, JMLR, and AutoML-Conf (2022-2025) finds that 98% never vary B/|A| as a controlled axis. On the same GDSC2 benchmark, changing only the budget reverses the ranking: at B=50, Greedy outperforms UCB by 0.050 Hit@1, while at B=100, UCB outperforms Greedy by 0.035. We capture this transition with the Portable Regime Score PRS=(B/|A|)(1-rho), where rho is the prior rank correlation and can be estimated from pilot contexts before the main comparison. Across 79 conditions spanning chemistry, drug-response biology, and HPO, a hierarchical model gives beta=0.50 (p=1.1e-9), and 19% of conditions fall in an equivalence zone where |advantage|<0.01 Hit@1. In five published reversal cases, PRS predicts the winner from pre-comparison observables. A No-Free-Leaderboard proposition explains why unconditional rankings are unstable: when CATE changes sign across regimes, the reported ATE becomes a function of benchmark mixture. RegimePlanner, which estimates rho online and switches acquisition accordingly, wins all 16 HPO-B search spaces at B=100 and exceeds the matched {Greedy,UCB} per-context oracle on GDSC2 by 18%. Pre-registered predictions achieve 27/40=67.5% overall accuracy and above 90% within EMA prior families. The practical protocol is simple: report B/|A|, rho, K, and metric alongside any claimed acquisition advantage.
翻译:已发表的迁移贝叶斯优化对比研究通常估计采集函数选择在隐式情境变量上的平均处理效应,而实践者需要根据其先验质量、预算比例和评价指标获取条件效应。本文对NeurIPS、ICML、ICLR、AISTATS、UAI、TMLR、JMLR及AutoML-Conf(2022-2025)中40篇迁移贝叶斯优化论文进行审查,发现98%的论文从未将B/|A|作为受控轴变量进行变化。在相同GDSC2基准测试中,仅改变预算即导致排序反转:当B=50时,Greedy算法以0.050的Hit@1指标优于UCB;而当B=100时,UCB以0.035的Hit@1指标反超。我们通过便携式情境评分PRS=(B/|A|)(1-rho)捕捉该转换规律,其中rho为先验秩相关系数,可在主对比分析前通过试点情境进行估计。在涵盖化学、药物反应生物学和超参数优化(HPO)的79个条件下,层次模型给出beta=0.50(p=1.1e-9),其中19%的条件落入|优势|<0.01 Hit@1的等价区间。在五例已发表的排序反转案例中,PRS通过对比前的可观测变量成功预测胜出算法。无免费排行榜命题解释了非条件排序不稳定的根本原因:当条件平均处理效应(CATE)随情境改变符号时,报告的平均处理效应(ATE)将成为基准混合比例的函数。RegimePlanner框架通过在线估计rho并相应切换采集函数,在B=100时横扫所有16个HPO-B搜索空间,并在GDSC2上以18%的幅度超越匹配的{Greedy, UCB}每情境最优组合。预注册预测实现40次中27次正确(67.5%)的整体准确率,在EMA先验族中准确率超过90%。实用协议简单明了:报告任何声称的采集优势时,须同时提供B/|A|、rho、K和评价指标。