In this paper, we propose a harmonized rotational gradient method, termed HRGrad, for simultaneously tackling multiscale time-dependent kinetic problems with varying small parameters. These parameters exhibit asymptotic transitions from microscopic to macroscopic physics, making it a challenging multi-task problem to solve over all ranges simultaneously. Solving tasks in different asymptotic regions often encounter gradient conflicts, which can lead to the failure of multi-task learning. To address this challenge, we explicitly encode a hidden representation of these parameters, ensuring that the corresponding solving tasks are serialized for simultaneous training. Furthermore, to mitigate gradient conflicts, we segment the prediction results to construct task losses and introduce a novel gradient alignment metric to ensure a positive dot product between the final update and each loss-specific gradient. This metric maintains consistent optimization rates for all task losses and dynamically adjusts gradient magnitudes based on conflict levels. Moreover, we provide a mathematical proof demonstrating the convergence of the HRGrad method, which is evaluated across a range of challenging asymptotic-preserving neural networks (APNNs) scenarios. We conduct an extensive set of experiments encompassing the Bhatnagar-Gross-Krook (BGK) equation and the linear transport equation in all ranges of Knudsen number. Our results indicate that HRGrad effectively overcomes the `failure modes' of APNNs in these problems.

翻译：本文提出一种名为HRGrad的调和旋转梯度方法，旨在同时求解含小参数变化的多尺度时变动力学问题。这些参数表现出从微观到宏观物理学的渐近过渡特性，使得在全尺度范围内同步求解成为具有挑战性的多任务问题。在不同渐近区域求解任务时，常遭遇梯度冲突，导致多任务学习失败。为解决这一挑战，我们显式编码了这些参数的隐藏表示，确保相应求解任务序列化以实现同步训练。同时，为缓解梯度冲突，我们对预测结果进行分段以构建任务损失，并引入新型梯度对齐度量，确保最终更新方向与每个特定损失的梯度方向保持正点积。该度量可维持所有任务损失的优化速率一致性，并根据冲突程度动态调整梯度幅值。此外，我们给出HRGrad方法收敛性的数学证明，并在多种具有挑战性的渐近保持神经网络（APNNs）场景中进行了验证。我们开展了涵盖Bhatnagar-Gross-Krook（BGK）方程及所有Knudsen数范围线性输运方程的广泛实验，结果表明HRGrad能有效克服这些问题的APNNs"失效模式"。