We propose a stochastic trust-region method for unconstrained nonconvex optimization that incorporates stochastic variance-reduced gradients (SVRG) to accelerate convergence. Unlike classical trust-region methods, the proposed algorithm relies solely on stochastic gradient information and does not require function value evaluations. The trust-region radius is adaptively adjusted based on a radius-control parameter and the stochastic gradient estimate. Under mild assumptions, we establish that the algorithm converges in expectation to a first-order stationary point. Moreover, the method achieves iteration and sample complexity bounds that match those of SVRG-based first-order methods, while allowing stochastic and potentially gradient-dependent second-order information. Extensive numerical experiments demonstrate that incorporating SVRG accelerates convergence, and that the use of trust-region methods and Hessian information further improves performance. We also highlight the impact of batch size and inner-loop length on efficiency, and show that the proposed method outperforms SGD and Adam on several machine learning tasks.

翻译：本文针对无约束非凸优化问题提出了一种随机信赖域方法，该方法结合随机方差缩减梯度（SVRG）以加速收敛。与经典信赖域方法不同，所提算法仅依赖随机梯度信息，无需进行函数值评估。信赖域半径根据半径控制参数和随机梯度估计值进行自适应调整。在温和假设下，我们证明该算法在期望意义下收敛至一阶稳定点。此外，该方法达到了与基于SVRG的一阶方法相匹配的迭代复杂度和样本复杂度界限，同时允许使用随机且可能依赖于梯度的二阶信息。大量数值实验表明，引入SVRG可加速收敛，而信赖域方法与海森矩阵信息的运用能进一步提升性能。我们还分析了批量大小和内循环长度对效率的影响，并证明所提方法在多个机器学习任务上优于SGD和Adam算法。