Data augmentation has been established as an efficacious approach to supplement useful information for low-resource datasets. Traditional augmentation techniques such as noise injection and image transformations have been widely used. In addition, generative data augmentation (GDA) has been shown to produce more diverse and flexible data. While generative adversarial networks (GANs) have been frequently used for GDA, they lack diversity and controllability compared to text-to-image diffusion models. In this paper, we propose TTIDA (Text-to-Text-to-Image Data Augmentation) to leverage the capabilities of large-scale pre-trained Text-to-Text (T2T) and Text-to-Image (T2I) generative models for data augmentation. By conditioning the T2I model on detailed descriptions produced by T2T models, we are able to generate photo-realistic labeled images in a flexible and controllable manner. Experiments on in-domain classification, cross-domain classification, and image captioning tasks show consistent improvements over other data augmentation baselines. Analytical studies in varied settings, including few-shot, long-tail, and adversarial, further reinforce the effectiveness of TTIDA in enhancing performance and increasing robustness.

翻译：数据增强已被证明是丰富低资源数据集有效信息的一种高效方法。传统的增强技术如噪声注入和图像变换已被广泛使用。此外，生成式数据增强（GDA）已展现出生成更多样化和灵活数据的能力。尽管生成对抗网络（GANs）常被用于GDA，但与文本到图像扩散模型相比，其多样性和可控性存在不足。本文提出TTIDA（文本到文本到图像数据增强）方法，利用大规模预训练的文本到文本（T2T）与文本到图像（T2I）生成模型进行数据增强。通过将T2I模型的条件设置为T2T模型生成的详细描述，我们能够以灵活可控的方式生成逼真的带标签图像。在域内分类、跨域分类及图像描述任务上的实验表明，该方法相较于其他数据增强基线方法取得了一致性改进。针对少样本、长尾分布及对抗性场景等不同设置的剖析研究，进一步验证了TTIDA在提升性能与增强鲁棒性方面的有效性。