This paper presents our proposed approach that won the first prize at the ICLR competition on Hardware Aware Efficient Training. The challenge is to achieve the highest possible accuracy in an image classification task in less than 10 minutes. The training is done on a small dataset of 5000 images picked randomly from CIFAR-10 dataset. The evaluation is performed by the competition organizers on a secret dataset with 1000 images of the same size. Our approach includes applying a series of technique for improving the generalization of ResNet-9 including: sharpness aware optimization, label smoothing, gradient centralization, input patch whitening as well as metalearning based training. Our experiments show that the ResNet-9 can achieve the accuracy of 88% while trained only on a 10% subset of CIFAR-10 dataset in less than 10 minuets

翻译：本文介绍了我们在ICLR硬件感知高效训练竞赛中获得一等奖的所提方法。该挑战要求在10分钟内以图像分类任务达到最高可能的准确率。训练使用从CIFAR-10数据集中随机选取的5000张图像组成的小数据集。评估由竞赛组织者对包含1000张同尺寸图像的秘密数据集进行。我们的方法包括应用一系列提升ResNet-9泛化能力的技术：锐度感知优化、标签平滑、梯度居中、输入斑块白化以及基于元学习的训练。实验表明，ResNet-9在仅使用CIFAR-10数据集10%子集训练且耗时不足10分钟的条件下，可达88%的准确率。