Zero-Shot Learning (ZSL) aims to recognize unseen classes by generalizing the knowledge, i.e., visual and semantic relationships, obtained from seen classes, where image augmentation techniques are commonly applied to improve the generalization ability of a model. However, this approach can also cause adverse effects on ZSL since the conventional augmentation techniques that solely depend on single-label supervision is not able to maintain semantic information and result in the semantic distortion issue consequently. In other words, image argumentation may falsify the semantic (e.g., attribute) information of an image. To take the advantage of image augmentations while mitigating the semantic distortion issue, we propose a novel ZSL approach by Harnessing Adversarial Samples (HAS). HAS advances ZSL through adversarial training which takes into account three crucial aspects: (1) robust generation by enforcing augmentations to be similar to negative classes, while maintaining correct labels, (2) reliable generation by introducing a latent space constraint to avert significant deviations from the original data manifold, and (3) diverse generation by incorporating attribute-based perturbation by adjusting images according to each semantic attribute's localization. Through comprehensive experiments on three prominent zero-shot benchmark datasets, we demonstrate the effectiveness of our adversarial samples approach in both ZSL and Generalized Zero-Shot Learning (GZSL) scenarios. Our source code is available at https://github.com/uqzhichen/HASZSL.

翻译：零样本学习旨在通过泛化从已知类别中获得的视觉与语义关系知识来识别未见类别，其中图像增强技术常被用于提升模型的泛化能力。然而，该方法也可能对零样本学习产生负面影响，因为仅依赖单标签监督的传统增强技术无法维持语义信息，进而导致语义失真问题。换言之，图像增强可能扭曲图像的语义（如属性）信息。为在利用图像增强优势的同时缓解语义失真问题，我们提出了一种利用对抗样本的新型零样本学习方法。该方法通过对抗训练推进零样本学习，并综合考虑三个关键方面：（1）鲁棒生成——强制增强样本与负类别相似，同时保持正确标签；（2）可靠生成——引入潜在空间约束以避免与原始数据流形产生显著偏离；（3）多样生成——基于属性定位的扰动生成，依据各语义属性的空间定位调整图像。通过在三个主流零样本基准数据集上的全面实验，我们证明了对抗样本方法在零样本学习和广义零样本学习场景中的有效性。我们的源代码已开源至 https://github.com/uqzhichen/HASZSL。