Generating high-quality synthetic text under differential privacy (DP) is critical for training and evaluating language models without compromising user privacy. Prior work on synthesizing DP datasets often fail to preserve key statistical attributes, suffer utility loss from the noise required by DP, and lack fine-grained control over generation. To address these challenges, we make two contributions. First, we introduce a hierarchical framework that decomposes DP synthetic text generation into two subtasks: feature learning and conditional text generation. This design explicitly incorporates learned features into the generation process and simplifies the end-to-end synthesis task. Through systematic ablations, we identify the most effective configuration: a rich tabular schema as feature, a DP tabular synthesizer, and a DP fine-tuned conditional generator, which we term ACTG (Attribute-Conditioned Text Generation). Second, we propose Anchored RL (ARL), a post-training method that improves the instruction-following ability of ACTG for conditional generation. ARL combines RL to boost control with an SFT anchor on best-of-$N$ data to prevent reward hacking. Together, these components form our end-to-end algorithm ACTG-ARL, which advances both the quality of DP synthetic text (+20% MAUVE over prior work) and the control of the conditional generator under strong privacy guarantees.

翻译：在差分隐私（DP）约束下生成高质量的合成文本，对于在不损害用户隐私的前提下训练和评估语言模型至关重要。现有合成DP数据集的工作往往难以保留关键统计属性，因DP所需的噪声而遭受效用损失，并且缺乏对生成的细粒度控制。为解决这些挑战，我们做出了两项贡献。首先，我们引入了一个分层框架，将DP合成文本生成分解为两个子任务：特征学习和条件文本生成。该设计明确地将学习到的特征纳入生成过程，并简化了端到端的合成任务。通过系统性的消融实验，我们确定了最有效的配置：以丰富的表格模式作为特征、一个DP表格合成器以及一个DP微调的条件生成器，我们将其称为ACTG（属性条件文本生成）。其次，我们提出了锚定强化学习（ARL），这是一种后训练方法，旨在提升ACTG在条件生成中的指令遵循能力。ARL结合了强化学习以增强控制，并利用基于最佳-$N$数据的SFT锚点来防止奖励破解。这些组件共同构成了我们的端到端算法ACTG-ARL，该算法在强隐私保证下，同时提升了DP合成文本的质量（相较于先前工作，MAUVE指标提升+20%）和条件生成器的控制能力。