Computing power has evolved into a foundational and indispensable resource in the area of deep learning, particularly in tasks such as Face Recognition (FR) model training on large-scale datasets, where multiple GPUs are often a necessity. Recognizing this challenge, some FR methods have started exploring ways to compress the fully-connected layer in FR models. Unlike other approaches, our observations reveal that without prompt scheduling of the learning rate (LR) during FR model training, the loss curve tends to exhibit numerous stationary subsequences. To address this issue, we introduce a novel LR scheduler leveraging Exponential Moving Average (EMA) and Haar Convolutional Kernel (HCK) to eliminate stationary subsequences, resulting in a significant reduction in converging time. However, the proposed scheduler incurs a considerable computational overhead due to its time complexity. To overcome this limitation, we propose FastFace, a fast-converging scheduler with negligible time complexity, i.e. O(1) per iteration, during training. In practice, FastFace is able to accelerate FR model training to a quarter of its original time without sacrificing more than 1% accuracy, making large-scale FR training feasible even with just one single GPU in terms of both time and space complexity. Extensive experiments validate the efficiency and effectiveness of FastFace. The code is publicly available at: https://github.com/amoonfana/FastFace

翻译：计算能力已成为深度学习领域的基础性且不可或缺的资源，特别是在大规模数据集上进行人脸识别模型训练等任务中，多GPU往往是必要条件。针对这一挑战，一些人脸识别方法已开始探索压缩模型中全连接层的途径。与其他方法不同，我们的观察表明，若在人脸识别模型训练期间未能及时调整学习率，损失曲线往往呈现大量平稳子序列。为解决此问题，我们提出一种新颖的学习率调度器，利用指数移动平均与哈尔卷积核消除平稳子序列，从而显著缩短收敛时间。然而，该调度器因其时间复杂度而带来可观的计算开销。为克服此局限，我们提出FastFace——一种具有可忽略时间复杂度（即每次迭代O(1)）的高效收敛调度器。实际应用中，FastFace能在精度损失不超过1%的前提下，将人脸识别模型训练速度提升至原始时间的四分之一，使得即使仅使用单个GPU，在大规模人脸识别训练的时间与空间复杂度层面均具备可行性。大量实验验证了FastFace的效率与有效性。代码已公开于：https://github.com/amoonfana/FastFace