Freehand sketches exhibit unique sparsity and abstraction, necessitating learning pipelines distinct from those designed for images. For sketch learning methods, the central objective is to fully exploit the effective information embedded in sketches. However, there is limited research on what constitutes effective sketch information, which in turn constrains the performance of existing approaches. To tackle this issue, we first proposed the Multi-Level Sketch Representation Scheme to identify the effective information. The scheme organizes sketch representation into three levels: sketch-level, stroke-level, and point-level. This design is based on the granularity of analytical elements, from coarse (sketch-level) to fine (point-level), thereby ensuring more comprehensive coverage of the sketch information. For each level, we conducted theoretical analyses and experimental evaluations to identify and validate the effective information. Building on the above studies, we developed SDGraph, a deep learning architecture designed to exploit the identified effective information across the three levels. SDGraph comprises two complementary modules: a Sparse Graph that treats strokes as nodes for sketch-level and stroke-level representation learning, and a Dense Graph that treats points as nodes for sketch-level and point-level representation learning. Both modules employ graph convolution along with down-sampling and up-sampling operations, enabling them to function as both encoder and decoder. Besides that, an information fusion module bridges the two graphs to further enhance feature extraction. SDGraph supports a wide range of sketch-related downstream tasks, achieving accuracy improvements of 1.15\% and 2.30\% over the state-of-the-art in classification and retrieval, respectively, and 32.93\% improvement in vector sketch generation quality.
翻译:手绘草图展现出独特的稀疏性与抽象性,需要不同于图像处理的学习流程。对于草图学习方法而言,核心目标在于充分挖掘草图中蕴含的有效信息。然而,关于何为有效草图信息的研究尚不充分,这反过来制约了现有方法的性能。为解决这一问题,我们首先提出了多层次草图表示方案以识别有效信息。该方案将草图表示组织为三个层次:草图级、笔画级和点级。这一设计基于分析元素的粒度,从粗粒度(草图级)到细粒度(点级),从而确保更全面地覆盖草图信息。针对每个层次,我们进行了理论分析与实验评估,以识别并验证有效信息。基于上述研究,我们开发了SDGraph——一种旨在利用三个层次中已识别有效信息的深度学习架构。SDGraph包含两个互补模块:将笔画视为节点以进行草图级与笔画级表示学习的稀疏图,以及将点视为节点以进行草图级与点级表示学习的稠密图。两个模块均采用图卷积配合下采样与上采样操作,使其能够同时作为编码器与解码器。此外,信息融合模块连接两个图以进一步增强特征提取。SDGraph支持广泛的草图相关下游任务,在分类与检索任务中分别比现有最优方法提升了1.15%和2.30%的准确率,并在矢量草图生成质量上实现了32.93%的提升。