Pattern images are everywhere in the digital and physical worlds, and tools to edit them are valuable. But editing pattern images is tricky: desired edits are often programmatic: structure-aware edits that alter the underlying program which generates the pattern. One could attempt to infer this underlying program, but current methods for doing so struggle with complex images and produce unorganized programs that make editing tedious. In this work, we introduce a novel approach to perform programmatic edits on pattern images. By using a pattern analogy -- a pair of simple patterns to demonstrate the intended edit -- and a learning-based generative model to execute these edits, our method allows users to intuitively edit patterns. To enable this paradigm, we introduce SplitWeave, a domain-specific language that, combined with a framework for sampling synthetic pattern analogies, enables the creation of a large, high-quality synthetic training dataset. We also present TriFuser, a Latent Diffusion Model (LDM) designed to overcome critical issues that arise when naively deploying LDMs to this task. Extensive experiments on real-world, artist-sourced patterns reveals that our method faithfully performs the demonstrated edit while also generalizing to related pattern styles beyond its training distribution.
翻译:模式图像在数字和物理世界中无处不在,编辑它们的工具具有重要价值。然而,编辑模式图像具有挑战性:期望的编辑通常是程序化的——即能够感知结构、改变生成模式底层程序的编辑。人们可以尝试推断这个底层程序,但现有方法在处理复杂图像时存在困难,且生成的程序结构混乱,使得编辑过程繁琐。本文提出了一种对模式图像执行程序化编辑的新方法。通过使用模式类比——即一对用于演示预期编辑意图的简单模式——以及基于学习的生成模型来执行这些编辑,我们的方法使用户能够直观地编辑模式。为实现这一范式,我们提出了SplitWeave,这是一种领域特定语言,结合用于采样合成模式类比的框架,能够创建大规模、高质量的合成训练数据集。我们还提出了TriFuser,这是一种潜在扩散模型(LDM),旨在解决将该模型简单应用于此任务时出现的关键问题。在真实世界艺术家创作的模式图像上进行的大量实验表明,我们的方法能够忠实执行所演示的编辑,同时还能泛化到训练分布之外的相关模式风格。