While diffusion models excel at image generation, their growing adoption raises critical concerns about copyright issues and model transparency. Existing attribution methods identify training examples influencing an entire image, but fall short in isolating contributions to specific elements, such as styles or objects, that are of primary concern to stakeholders. To address this gap, we introduce concept-level attribution through a novel method called Concept-TRAK, which extends influence functions with a key innovation: specialized training and utility loss functions designed to isolate concept-specific influences rather than overall reconstruction quality. We evaluate Concept-TRAK on novel concept attribution benchmarks using Synthetic and CelebA-HQ datasets, as well as the established AbC benchmark, showing substantial improvements over prior methods in concept-level attribution scenarios. We further demonstrate its versatility on real-world text-to-image generation with compositional and multi-concept prompts.
翻译:尽管扩散模型在图像生成方面表现出色,但其日益广泛的应用引发了关于版权问题和模型透明度的关键关切。现有的归因方法能够识别影响整幅图像的训练样本,但在分离对特定元素(如风格或对象)的贡献方面存在不足,而这些元素正是利益相关者最为关注的。为弥补这一不足,我们通过一种名为概念-TRAK的新方法引入了概念级归因,该方法扩展了影响函数,其核心创新在于:设计了专门的训练和效用损失函数,旨在分离概念特定的影响,而非整体重建质量。我们使用合成数据集和CelebA-HQ数据集,以及已建立的AbC基准,在全新的概念归因基准上评估了概念-TRAK,结果显示其在概念级归因场景中相较于先前方法有显著提升。我们进一步通过组合式和多概念提示,在真实世界的文本到图像生成任务中展示了其多功能性。