Coupled gradient descent--where the update of one parameter block depends on another--underlies bilevel optimization, two-time-scale stochastic approximation, and adversarial training. When the coupled Jacobian is block-triangular, asymptotic stability is governed by the spectral radii of the diagonal blocks, yet transient amplification before convergence can be arbitrarily large due to non-normality. We develop a sharp pseudospectral theory for such block-triangular Jacobians, proving that the Kreiss constant satisfies $K(J) \leq 2/(1-γ) + \|C\|/(4(1-γ))$ when the diagonal blocks are symmetric with spectral radii at most $γ< 1$, and we establish matching minimax lower bounds. We characterize the critical coupling threshold for spectral instability and extend the analysis to nearly self-referential systems via a Neumann-series perturbation framework. As a consequence, we obtain a finite-horizon iteration-complexity bound of $O(K(J)^2 \log(1/δ))$ for stochastic coupled descent. Framed as scaling laws for non-stationary two-time-scale optimization, our results expose a non-asymptotic, instance-dependent regime of high-dimensional learning dynamics that is invisible to spectral-radius analysis. Experiments on linear-quadratic problems, IQC-based comparisons, and neural-network training confirm the theory.

翻译：耦合梯度下降——其中一个参数块的更新依赖于另一个——构成了双层优化、双时间尺度随机逼近和对抗性训练的基础。当耦合雅可比矩阵为分块三角阵时，渐近稳定性由对角块的谱半径决定，但由于非正态性，收敛前的瞬态放大可能任意大。我们针对此类分块三角雅可比矩阵建立了精细的伪谱理论，证明当对角块对称且谱半径不超过γ<1时，Kreiss常数满足$K(J) ≤ 2/(1-γ) + \|C\|/(4(1-γ))$，并建立了匹配的极小化极大下界。我们刻画了谱不稳定性的临界耦合阈值，并借助诺伊曼级数扰动框架将分析推广至近自指系统。作为推论，我们得到了随机耦合下降的有限步迭代复杂度上界$O(K(J)^2 \log(1/δ))$。从非平稳双时间尺度优化的标度律角度审视，我们的结果揭示了一个谱半径分析无法捕捉的非渐近、与实例相关的高维学习动力学区域。在线性二次型问题、基于IQC的比较以及神经网络训练上的实验验证了该理论。