In this paper, we explore the unique modality of sketch for explainability, emphasising the profound impact of human strokes compared to conventional pixel-oriented studies. Beyond explanations of network behavior, we discern the genuine implications of explainability across diverse downstream sketch-related tasks. We propose a lightweight and portable explainability solution -- a seamless plugin that integrates effortlessly with any pre-trained model, eliminating the need for re-training. Demonstrating its adaptability, we present four applications: highly studied retrieval and generation, and completely novel assisted drawing and sketch adversarial attacks. The centrepiece to our solution is a stroke-level attribution map that takes different forms when linked with downstream tasks. By addressing the inherent non-differentiability of rasterisation, we enable explanations at both coarse stroke level (SLA) and partial stroke level (P-SLA), each with its advantages for specific downstream tasks.
翻译:本文探索了草图作为一种独特的可解释性模态,强调了人类笔画相较于传统像素导向研究的深远影响。超越网络行为解释,我们辨别了可解释性在多种与草图相关的下游任务中的实际含义。我们提出了一种轻量级且可移植的可解释性解决方案——一个能无缝集成到任何预训练模型中的即插即用插件,无需重新训练。通过展示其适应性,我们呈现了四种应用:广为研究的检索与生成,以及全新概念的辅助绘图和草图对抗攻击。该解决方案的核心是一个与下游任务关联时呈现不同形式的笔画级归因图。通过解决光栅化固有的不可微性问题,我们实现了粗笔画级(SLA)和部分笔画级(P-SLA)两个层面的解释,每种对特定下游任务各有优势。