The drawing order of a sketch records how it is created stroke-by-stroke by a human being. For graphic sketch representation learning, recent studies have injected sketch drawing orders into graph edge construction by linking each patch to another in accordance to a temporal-based nearest neighboring strategy. However, such constructed graph edges may be unreliable, since a sketch could have variants of drawings. In this paper, we propose a variant-drawing-protected method by equipping sketch patches with context-aware positional encoding (PE) to make better use of drawing orders for learning graphic sketch representation. Instead of injecting sketch drawings into graph edges, we embed these sequential information into graph nodes only. More specifically, each patch embedding is equipped with a sinusoidal absolute PE to highlight the sequential position in the drawing order. And its neighboring patches, ranked by the values of self-attention scores between patch embeddings, are equipped with learnable relative PEs to restore the contextual positions within a neighborhood. During message aggregation via graph convolutional networks, a node receives both semantic contents from patch embeddings and contextual patterns from PEs by its neighbors, arriving at drawing-order-enhanced sketch representations. Experimental results indicate that our method significantly improves sketch healing and controllable sketch synthesis.
翻译:草图的绘制顺序记录了人类一笔一划创建它的过程。在图形草图表示学习中,最近的研究通过将每个补丁与另一个补丁按照基于时间的最近邻策略连接起来,将草图绘制顺序注入到图边构建中。然而,由于草图可能存在多种绘制变体,这种构建的图边可能不可靠。在本文中,我们提出了一种保护绘制变体的方法,通过为草图补丁配备上下文感知的位置编码,从而更好地利用绘制顺序来学习图形草图表示。我们不将草图绘制信息注入图边,而是仅将这些序列信息嵌入到图节点中。具体来说,每个补丁嵌入都被赋予正弦绝对位置编码,以突出其在绘制顺序中的序列位置。而其相邻补丁(根据补丁嵌入之间的自注意力得分值排序)则被赋予可学习的相对位置编码,以恢复邻域内的上下文位置。在通过图卷积网络进行消息聚合时,节点从其邻居处同时接收补丁嵌入的语义内容和位置编码的上下文模式,从而获得增强绘制顺序的草图表示。实验结果表明,我们的方法显著改善了草图的修复和可控的草图合成。