This paper investigates the weaknesses of image watermarking techniques. We present WAVES (Watermark Analysis Via Enhanced Stress-testing), a novel benchmark for assessing watermark robustness, overcoming the limitations of current evaluation methods.WAVES integrates detection and identification tasks, and establishes a standardized evaluation protocol comprised of a diverse range of stress tests. The attacks in WAVES range from traditional image distortions to advanced and novel variations of diffusive, and adversarial attacks. Our evaluation examines two pivotal dimensions: the degree of image quality degradation and the efficacy of watermark detection after attacks. We develop a series of Performance vs. Quality 2D plots, varying over several prominent image similarity metrics, which are then aggregated in a heuristically novel manner to paint an overall picture of watermark robustness and attack potency. Our comprehensive evaluation reveals previously undetected vulnerabilities of several modern watermarking algorithms. We envision WAVES as a toolkit for the future development of robust watermarking systems. The project is available at https://wavesbench.github.io/

翻译：本文研究了图像水印技术的弱点。我们提出了WAVES（基于增强压力测试的水印分析），这是一个用于评估水印鲁棒性的新型基准，克服了当前评估方法的局限性。WAVES整合了检测与识别任务，并建立了一套包含多种压力测试的标准化评估协议。WAVES中的攻击手段涵盖从传统图像失真到先进且新颖的扩散性攻击及对抗性攻击变种。我们的评估聚焦两个关键维度：攻击后图像质量的退化程度以及水印检测的有效性。我们绘制了一系列性能与质量的二维图表，这些图表基于多种主流图像相似度度量指标，并通过一种启发式的新颖方法进行聚合，从而全面描绘水印鲁棒性与攻击强度的全景。我们的综合评估揭示了多个现代水印算法此前未被发现的漏洞。我们期望WAVES能成为未来开发鲁棒水印系统的工具包。相关项目可在https://wavesbench.github.io/获取。