Security classifiers, designed to detect malicious content in computer systems and communications, can underperform when provided with insufficient training data. In the security domain, it is often easy to find samples of the negative (benign) class, and challenging to find enough samples of the positive (malicious) class to train an effective classifier. This study evaluates the application of natural language text generators to fill this data gap in multiple security-related text classification tasks. We describe a variety of previously-unexamined language-model fine-tuning approaches for this purpose and consider in particular the impact of disproportionate class-imbalances in the training set. Across our evaluation using three state-of-the-art classifiers designed for offensive language detection, review fraud detection, and SMS spam detection, we find that models trained with GPT-3 data augmentation strategies outperform both models trained without augmentation and models trained using basic data augmentation strategies already in common usage. In particular, we find substantial benefits for GPT-3 data augmentation strategies in situations with severe limitations on known positive-class samples.

翻译：安全分类器旨在检测计算机系统及通信中的恶意内容，但当训练数据不足时可能性能欠佳。在安全领域，负例（良性）样本通常容易获取，而难以收集足够数量的正例（恶意）样本以训练有效分类器。本研究评估了自然语言文本生成器在多项安全相关文本分类任务中填补这一数据缺口的应用。我们针对此目的描述了多种此前未系统研究的语言模型微调方法，并特别关注训练集中比例悬殊的类别不平衡问题的影响。通过采用面向攻击性语言检测、评论欺诈检测及短信垃圾检测设计的三种最先进分类器进行的评估表明，采用GPT-3数据增强策略训练的模型在性能上全面优于未使用增强训练的模型以及采用现有常用基础数据增强策略训练的模型。特别值得注意的是，当已知正例样本严重匮乏时，GPT-3数据增强策略展现出显著优势。