Neural networks (NNs) often have critical behavioural trade-offs that are set at design time with hyperparameters-such as reward weights in reinforcement learning or quantile targets in regression. Post-deployment, however, user preferences can evolve, making initial settings undesirable, necessitating potentially expensive retraining. To circumvent this, we introduce the task of Hyperparameter Trajectory Inference (HTI): to learn, from observed data, how a NN's conditional output distribution changes with its hyperparameters, and construct a surrogate model that approximates the NN at unobserved hyperparameter settings. HTI requires extending existing trajectory inference approaches to incorporate conditions, exacerbating the challenge of ensuring inferred paths are feasible. We propose an approach based on conditional Lagrangian optimal transport, jointly learning the Lagrangian function governing hyperparameter-induced dynamics along with the associated optimal transport maps and geodesics between observed marginals, which form the surrogate model. We incorporate inductive biases based on the manifold hypothesis and least-action principles into the learned Lagrangian, improving surrogate model feasibility. We empirically demonstrate that our approach reconstructs NN outputs across various hyperparameter spectra better than other alternatives.

翻译：神经网络（NNs）在设计时通常通过超参数设定关键的行为权衡——例如强化学习中的奖励权重或回归中的分位数目标。然而，部署后用户偏好可能发生变化，使得初始设定不再理想，从而可能需要代价高昂的重新训练。为避免此问题，我们引入了超参数轨迹推断（HTI）任务：从观测数据中学习神经网络的**条件输出分布**如何随其超参数变化，并构建一个在未观测超参数设定下近似该神经网络的**代理模型**。HTI要求将现有轨迹推断方法扩展至包含条件的情形，这加剧了确保推断路径可行的挑战。我们提出了一种基于**条件拉格朗日最优传输**的方法，该方法联合学习支配超参数诱导动力学的拉格朗日函数，以及观测边缘分布之间的关联最优传输映射与测地线，这些共同构成了代理模型。我们将基于流形假设和最小作用量原理的归纳偏置融入所学拉格朗日函数中，从而提升了代理模型的可行性。我们通过实验证明，我们的方法在多种超参数谱下重建神经网络输出的效果优于其他替代方案。