Faithful text rendering remains a persistent weakness of large text-to-image generative models, as it requires both semantic instruction following and fine-grained glyph-level structure. Prior methods often improve this ability through architecture-specific modules or encoder modifications, which complicate deployment across foundation models. We study text rendering as a post-training preference-alignment problem and propose TextAlign, a non-invasive framework that keeps the generator architecture unchanged. The key component is a hierarchical vision-language model (VLM)-based reward that decomposes rendering errors into global, word, and glyph levels, then converts binary defect judgments into a scalar preference signal. The resulting signal supports both Group Relative Policy Optimization (GRPO) and Direct Preference Optimization (DPO). Experiments on FLUX.1-dev and Z-Image-Turbo show consistent gains in OCR-based text accuracy without degrading general generation quality. Compared with strong foundation and text-rendering baselines, including SD3.5, Qwen-Image, AnyText, and TextDiffuser, these results indicate that reward design offers a scalable alternative to model redesign for improving text rendering.

翻译：[translated abstract in Chinese] 忠实的文本渲染仍然是大型文本到图像生成模型持续存在的弱点，因为它既需要遵循语义指令，又需要实现细粒度字形级结构。先前的方法通常通过特定于架构的模块或编码器修改来提升该能力，但这会复杂化跨基础模型的部署。我们将文本渲染研究为一个训练后的偏好对齐问题，并提出了TextAlign——一种保持生成器架构不变的非侵入式框架。其关键组件是一个基于分层视觉语言模型（VLM）的奖励机制，该机制将渲染错误分解为全局、单词和字形三个层级，然后将二元缺陷判断转换为标量偏好信号。该信号既支持群体相对策略优化（GRPO），也支持直接偏好优化（DPO）。在FLUX.1-dev和Z-Image-Turbo上的实验表明，该方法在不降低通用生成质量的情况下，在基于OCR的文本准确性上取得了一致提升。与包括SD3.5、Qwen-Image、AnyText和TextDiffuser在内的强大基础模型和文本渲染基线相比，这些结果表明，奖励设计为改进文本渲染提供了一种可扩展的替代方案，以替代模型重构。