Computational text classification is a challenging task, especially for multi-dimensional social constructs. Recently, there has been increasing discussion that synthetic training data could enhance classification by offering examples of how these constructs are represented in texts. In this paper, we systematically examine the potential of theory-driven synthetic training data for improving the measurement of social constructs. In particular, we explore how researchers can transfer established knowledge from measurement instruments in the social sciences, such as survey scales or annotation codebooks, into theory-driven generation of synthetic data. Using two studies on measuring sexism and political topics, we assess the added value of synthetic training data for fine-tuning text classification models. Although the results of the sexism study were less promising, our findings demonstrate that synthetic data can be highly effective in reducing the need for labeled data in political topic classification. With only a minimal drop in performance, synthetic data allows for substituting large amounts of labeled data. Furthermore, theory-driven synthetic data performed markedly better than data generated without conceptual information in mind.

翻译：计算文本分类是一项具有挑战性的任务，尤其对于多维度的社会建构而言。近期，越来越多的讨论认为，合成训练数据可以通过提供这些建构在文本中如何被表征的示例来增强分类性能。本文系统性地考察了理论驱动的合成训练数据在改进社会建构测量方面的潜力。具体而言，我们探讨了研究者如何将社会科学中测量工具（如调查量表或标注手册）的成熟知识转化为理论驱动的合成数据生成。通过两项关于性别歧视和政治议题测量的研究，我们评估了合成训练数据在微调文本分类模型中的附加价值。尽管性别歧视研究的结果不太理想，但我们的发现表明，合成数据在政治议题分类中可以显著减少对标注数据的需求，效果显著。在性能仅轻微下降的情况下，合成数据能够替代大量的标注数据。此外，理论驱动的合成数据表现明显优于未考虑概念信息生成的数据。