Training a linear classifier or lightweight model on top of pretrained vision model outputs, so-called 'frozen features', leads to impressive performance on a number of downstream few-shot tasks. Currently, frozen features are not modified during training. On the other hand, when networks are trained directly on images, data augmentation is a standard recipe that improves performance with no substantial overhead. In this paper, we conduct an extensive pilot study on few-shot image classification that explores applying data augmentations in the frozen feature space, dubbed 'frozen feature augmentation (FroFA)', covering twenty augmentations in total. Our study demonstrates that adopting a deceptively simple pointwise FroFA, such as brightness, can improve few-shot performance consistently across three network architectures, three large pretraining datasets, and eight transfer datasets.
翻译:在预训练视觉模型输出上训练线性分类器或轻量级模型(即所谓的“冻结特征”),能够在多项下游少样本任务中取得显著性能。目前,冻结特征在训练过程中保持不变。另一方面,当网络直接在图像上训练时,数据增强是一种标准方法,能在不增加大量开销的情况下提升性能。本文针对少样本图像分类开展了一项广泛的先导研究,探索在冻结特征空间中应用数据增强,称为“冻结特征增强(FroFA)”,共涵盖二十种增强方法。我们的研究表明,采用看似简单的逐点冻结特征增强(如亮度增强),能在三种网络架构、三个大型预训练数据集和八个迁移数据集上持续提升少样本性能。