Second-order methods promise faster convergence but are rarely used in practice because Hessian computations and decompositions are far more expensive than gradients. We propose a \emph{split-client} framework where gradients and curvature are computed asynchronously by separate clients. This abstraction captures realistic delays and inexact Hessian updates while avoiding the manual tuning required by Lazy Hessian methods. Focusing on cubic regularization, we show that our approach retains strong convergence guarantees and achieves a provable wall-clock speedup of order $\sqrt{\tau}$, where $\tau$ is the relative time needed to compute and decompose the Hessian compared to a gradient step. Since $\tau$ can be orders of magnitude larger than one in high-dimensional problems, this improvement is practically significant. Experiments on synthetic and real datasets confirm the theory: asynchronous curvature consistently outperforms vanilla and Lazy Hessian baselines, while maintaining second-order accuracy.
翻译:二阶方法虽能提供更快的收敛速度,但在实际中却很少使用,因为海森矩阵的计算与分解远比梯度计算昂贵。我们提出一种\emph{拆分客户端}框架,其中梯度和曲率由不同的客户端异步计算。该抽象方法捕捉了实际的延迟和不精确的海森矩阵更新,同时避免了惰性海森方法所需的手动调参。聚焦于三次正则化,我们证明了该方法保持了强大的收敛保证,并实现了阶为$\sqrt{\tau}$的可证明的挂钟加速,其中$\tau$是相对于梯度步进计算和分解海森矩阵所需的相对时间。由于在高维问题中$\tau$可能比一大几个数量级,这一改进具有实际意义。在合成和真实数据集上的实验证实了该理论:异步曲率计算持续优于朴素和惰性海森基线方法,同时保持了二阶精度。