A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multi-objective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks with multi-dimensional regularization losses.

翻译：本文提出了一种概率图模型，用于联合建模模型参数与乘子的演化过程，其基于超体积的似然函数旨在促进结构风险最小化中的多目标下降。我们通过一个具有时变乘子的代理单目标惩罚损失来解决多目标模型参数优化问题，这等价于对损失地形进行在线调度。多目标下降目标依据帕累托支配关系，被分层分解为一系列具有收缩边界的约束优化子问题。该边界作为底层乘子控制器的设定点，通过各损失项的输出反馈来调度损失地形。我们的方法形成了模型参数动态的闭环，规避了现有多目标深度学习方法中过高的内存需求和额外计算负担，并在具有多维正则化损失的领域泛化任务上验证了其对控制器超参数变化的鲁棒性。