In scientific machine learning, $Δ$-learning trains models on residual errors relative to approximate baselines, assuming that smaller residual scales inherently yield better downstream accuracy. Here, we demonstrate that residual scale alone is an insufficient heuristic for learnability. For molecular total energies, replacing a linear baseline with a neural network fitted to the same elemental counts reduced the residual scale, yet more than doubled GemNet's prediction error. Across local structural descriptors, more flexible baselines produced smaller residuals that varied sharply between structurally similar molecules. By contrast, semi-empirical baselines left larger residuals, yet yielded lower test error and improved out-of-domain transferability. Across the targets studied, scale-normalized graph Dirichlet roughness tracked scale-normalized prediction error, suggesting its use as a pre-training diagnostic for target selection. These results motivate baseline complementarity as a target-design principle in scientific machine learning: a useful baseline captures contributions the downstream architecture struggles to predict.
翻译:暂无翻译