We propose Dataset Reinforcement, a strategy to improve a dataset once such that the accuracy of any model architecture trained on the reinforced dataset is improved at no additional training cost for users. We propose a Dataset Reinforcement strategy based on data augmentation and knowledge distillation. Our generic strategy is designed based on extensive analysis across CNN- and transformer-based models and performing large-scale study of distillation with state-of-the-art models with various data augmentations. We create a reinforced version of the ImageNet training dataset, called ImageNet+, as well as reinforced datasets CIFAR-100+, Flowers-102+, and Food-101+. Models trained with ImageNet+ are more accurate, robust, and calibrated, and transfer well to downstream tasks (e.g., segmentation and detection). As an example, the accuracy of ResNet-50 improves by 1.7% on the ImageNet validation set, 3.5% on ImageNetV2, and 10.0% on ImageNet-R. Expected Calibration Error (ECE) on the ImageNet validation set is also reduced by 9.9%. Using this backbone with Mask-RCNN for object detection on MS-COCO, the mean average precision improves by 0.8%. We reach similar gains for MobileNets, ViTs, and Swin-Transformers. For MobileNetV3 and Swin-Tiny we observe significant improvements on ImageNet-R/A/C of up to 10% improved robustness. Models pretrained on ImageNet+ and fine-tuned on CIFAR-100+, Flowers-102+, and Food-101+, reach up to 3.4% improved accuracy.

翻译：我们提出数据集强化（Dataset Reinforcement）策略，旨在通过一次性改进数据集，使得在该强化数据集上训练的任何模型架构的用户无需额外训练成本即可提升精度。该策略基于数据增强与知识蒸馏，通过对卷积神经网络和Transformer基础模型进行广泛分析，并结合多种数据增强手段与先进模型的蒸馏大规模研究而设计。我们创建了ImageNet训练集的强化版本（称为ImageNet+），以及CIFAR-100+、Flowers-102+和Food-101+等强化数据集。基于ImageNet+训练的模型在精度、鲁棒性和校准性方面均表现更优，并能良好迁移至下游任务（如分割与检测）。例如，ResNet-50在ImageNet验证集上的精度提升1.7%，在ImageNetV2上提升3.5%，在ImageNet-R上提升10.0%；ImageNet验证集上的期望校准误差（ECE）降低9.9%。以该模型为骨干网络并采用Mask-RCNN进行MS-COCO目标检测时，平均精度均值（mAP）提升0.8%。我们在MobileNet、ViT和Swin-Transformer上取得了类似收益：对于MobileNetV3和Swin-Tiny，在ImageNet-R/A/C上的鲁棒性提升高达10%。基于ImageNet+预训练并在CIFAR-100+、Flowers-102+和Food-101+上微调的模型，精度最高提升3.4%。