This paper addresses the ethical concerns arising from the use of unauthorized public data in deep learning models and proposes a novel solution. Specifically, building on the work of Huang et al. (2021), we extend their bi-level optimization approach to generate unlearnable text using a gradient-based search technique. However, although effective, this approach faces practical limitations, including the requirement of batches of instances and model architecture knowledge that is not readily accessible to ordinary users with limited access to their own data. Furthermore, even with semantic-preserving constraints, unlearnable noise can alter the text's semantics. To address these challenges, we extract simple patterns from unlearnable text produced by bi-level optimization and demonstrate that the data remains unlearnable for unknown models. Additionally, these patterns are not instance- or dataset-specific, allowing users to readily apply them to text classification and question-answering tasks, even if only a small proportion of users implement them on their public content. We also open-source codes to generate unlearnable text and assess unlearnable noise to benefit the public and future studies.

翻译：本文针对深度学习模型使用未经授权的公开数据所引发的伦理问题，提出了一种新颖的解决方案。具体而言，基于Huang等人（2021）的工作，我们扩展了其双层优化方法，通过梯度搜索技术生成不可学习文本。然而，尽管该方法有效，但在实际应用中存在局限性，包括需要批次实例以及模型架构知识，而这些对于仅能有限访问自身数据的普通用户而言难以获取。此外，即使采用语义保持约束，不可学习噪声仍可能改变文本的语义。为解决这些挑战，我们从双层优化生成的不可学习文本中提取出简单模式，并证明这些数据对未知模型仍保持不可学习性。同时，这些模式不依赖于特定实例或数据集，使得用户可将其直接应用于文本分类和问答任务，即便只有少量用户在其公开内容中实施这些模式。我们还开源了生成不可学习文本及评估不可学习噪声的代码，以惠及公众及未来研究。