A cursory reading of the literature suggests that we have made a lot of progress in designing effective adversarial defenses for Graph Neural Networks (GNNs). Yet, the standard methodology has a serious flaw - virtually all of the defenses are evaluated against non-adaptive attacks leading to overly optimistic robustness estimates. We perform a thorough robustness analysis of 7 of the most popular defenses spanning the entire spectrum of strategies, i.e., aimed at improving the graph, the architecture, or the training. The results are sobering - most defenses show no or only marginal improvement compared to an undefended baseline. We advocate using custom adaptive attacks as a gold standard and we outline the lessons we learned from successfully designing such attacks. Moreover, our diverse collection of perturbed graphs forms a (black-box) unit test offering a first glance at a model's robustness.

翻译：对文献的粗略回顾表明，我们在为图神经网络（GNNs）设计有效对抗性防御方面取得了显著进展。然而，标准方法存在一个严重缺陷——几乎所有防御措施均基于非自适应攻击进行评估，导致对鲁棒性的估计过于乐观。我们对涵盖全策略谱系的7种最流行防御方法（即旨在改进图结构、架构或训练过程的防御）进行了彻底的鲁棒性分析。结果令人警醒——与未设防的基线相比，大多数防御方法仅表现出微乎其微的提升，甚至毫无改善。我们主张将定制化自适应攻击作为黄金标准，并总结了成功设计此类攻击的经验教训。此外，我们收集的多样化扰动图构成了一个（黑盒）单元测试，可为模型的鲁棒性提供初步评估。