Training deep computer vision models requires manual oversight or hyperparameter tuning of the learning rate (LR) schedule. While existing adaptive optimizers schedule the LR automatically, they suffer from computational and memory overhead, incompatibility with regularization, and suboptimal LR choices. In this work, we introduce the ZENITH (Zero-overhead Evolution using Norm-Informed Training History) optimizer, which adapts the LR using the temporal evolution of the gradient norm. Image classification experiments spanning 6 CNN architectures and 6 benchmarks demonstrate that ZENITH achieves higher test accuracy in lower wall-clock time than baselines. It also yielded superior mAP in object detection, keypoint detection, and instance segmentation on MS COCO using the R-CNN family of models. Furthermore, its compatibility with regularization enables even better generalization.

翻译：训练深度计算机视觉模型通常需要人工监控或对学习率（LR）调度进行超参数调优。尽管现有的自适应优化器能够自动调度学习率，但它们存在计算与内存开销大、与正则化方法不兼容以及学习率选择欠佳等问题。本研究提出ZENITH（基于范数训练历史演化的零开销优化器），该优化器利用梯度范数的时间演化特性自适应调整学习率。在涵盖6种CNN架构和6个基准测试的图像分类实验中，ZENITH在更短的挂钟时间内实现了比基线方法更高的测试准确率。在使用R-CNN系列模型对MS COCO数据集进行目标检测、关键点检测和实例分割任务时，该方法也取得了更优的mAP指标。此外，其与正则化方法的兼容性进一步提升了模型的泛化能力。