Personalization techniques for large text-to-image (T2I) models allow users to incorporate new concepts from reference images. However, existing methods primarily rely on textual descriptions, leading to limited control over customized images and failing to support fine-grained and local editing (e.g., shape, pose, and details). In this paper, we identify sketches as an intuitive and versatile representation that can facilitate such control, e.g., contour lines capturing shape information and flow lines representing texture. This motivates us to explore a novel task of sketch concept extraction: given one or more sketch-image pairs, we aim to extract a special sketch concept that bridges the correspondence between the images and sketches, thus enabling sketch-based image synthesis and editing at a fine-grained level. To accomplish this, we introduce CustomSketching, a two-stage framework for extracting novel sketch concepts. Considering that an object can often be depicted by a contour for general shapes and additional strokes for internal details, we introduce a dual-sketch representation to reduce the inherent ambiguity in sketch depiction. We employ a shape loss and a regularization loss to balance fidelity and editability during optimization. Through extensive experiments, a user study, and several applications, we show our method is effective and superior to the adapted baselines.

翻译：大型文生图模型的个性化技术允许用户从参考图像中融入新概念。然而，现有方法主要依赖文本描述，导致对定制图像的控制能力有限，且无法支持细粒度和局部编辑（如形状、姿态和细节）。本文提出将草图视为一种直观且多用途的表征形式，能够促进此类控制——例如捕捉形状信息的轮廓线和表征纹理的流线。这促使我们探索一项新任务——草图概念提取：给定一对或多对草图-图像对，旨在提取一种特殊的草图概念，该概念可桥接图像与草图之间的对应关系，从而实现细粒度的草图驱动图像合成与编辑。为实现该目标，我们提出CustomSketching——一个用于提取新颖草图概念的两阶段框架。考虑到物体通常可通过描绘整体形状的轮廓线和刻画内部细节的附加笔触来表征，我们引入双草图表征以减少草图描绘中固有的歧义性。在优化过程中，我们采用形状损失和正则化损失来平衡保真度与可编辑性。通过大量实验、用户研究及多项应用验证，本方法优于经适配的基线方法。