This paper proposes a method for hiding the least-important samples during the training of deep neural networks to increase efficiency, i.e., to reduce the cost of training. Using information about the loss and prediction confidence during training, we adaptively find samples to exclude in a given epoch based on their contribution to the overall learning process, without significantly degrading accuracy. We explore the converge properties when accounting for the reduction in the number of SGD updates. Empirical results on various large-scale datasets and models used directly in image classification and segmentation show that while the with-replacement importance sampling algorithm performs poorly on large datasets, our method can reduce total training time by up to 22% impacting accuracy only by 0.4% compared to the baseline. Code available at https://github.com/TruongThaoNguyen/kakurenbo

翻译：本文提出一种在深度神经网络训练过程中隐藏最不重要样本的方法，旨在提升效率，即降低训练成本。通过利用训练过程中的损失与预测置信度信息，我们根据样本对整体学习过程的贡献，自适应地确定每个训练轮次中需排除的样本，同时不显著降低准确性。我们探索了在减少随机梯度下降更新次数情况下的收敛特性。直接在图像分类与分割任务中使用多种大规模数据集和模型的实验结果表明，当有放回重要性采样算法在大数据集上表现不佳时，我们的方法可将总训练时间减少高达22%，而相较于基线仅影响0.4%的准确性。代码地址：https://github.com/TruongThaoNguyen/kakurenbo