Sabre is a defense to adversarial examples that was accepted at IEEE S&P 2024. We first reveal significant flaws in the evaluation that point to clear signs of gradient masking. We then show the cause of this gradient masking: a bug in the original evaluation code. By fixing a single line of code in the original repository, we reduce Sabre's robust accuracy to 0%. In response to this, the authors modify the defense and introduce a new defense component not described in the original paper. But this fix contains a second bug; modifying one more line of code reduces robust accuracy to below baseline levels.

翻译：Sabre是一种对抗样本防御方法，已被IEEE S&P 2024会议接收。我们首先揭示了其评估中存在的显著缺陷，这些缺陷明确指向梯度掩蔽的迹象。继而，我们展示了梯度掩蔽的根源：原始评估代码中的一个错误。通过修复原始仓库中的一行代码，我们将Sabre的鲁棒准确率降至0%。针对此问题，作者修改了防御方案并引入了一项原始论文中未描述的新防御组件。然而，该修复包含第二个错误：再修改一行代码即可使鲁棒准确率降至基准水平以下。