Nowadays, neural-network-based image- and video-quality metrics perform better than traditional methods. However, they also became more vulnerable to adversarial attacks that increase metrics' scores without improving visual quality. The existing benchmarks of quality metrics compare their performance in terms of correlation with subjective quality and calculation time. Nonetheless, the adversarial robustness of image-quality metrics is also an area worth researching. This paper analyses modern metrics' robustness to different adversarial attacks. We adapted adversarial attacks from computer vision tasks and compared attacks' efficiency against 15 no-reference image- and video-quality metrics. Some metrics showed high resistance to adversarial attacks, which makes their usage in benchmarks safer than vulnerable metrics. The benchmark accepts submissions of new metrics for researchers who want to make their metrics more robust to attacks or to find such metrics for their needs. The latest results can be found online: https://videoprocessing.ai/benchmarks/metrics-robustness.html.

翻译：当前，基于神经网络的图像与视频质量指标虽然优于传统方法，但其更易遭受对抗攻击——此类攻击可在不提升视觉质量的情况下提高指标得分。现有质量指标基准测试主要对比其与主观质量的关联性及计算速度，而图像质量指标的对抗鲁棒性同样是值得研究的重要领域。本文分析了现代指标对不同对抗攻击的鲁棒性。我们借鉴计算机视觉任务中的对抗攻击方法，系统比较了这些攻击对15种无参考图像与视频质量指标的攻击效率。结果表明，部分指标对对抗攻击具有高鲁棒性，这使得在基准测试中使用它们比易受攻击的指标更为可靠。本基准测试接受研究人员提交新型指标，以便研究者针对攻击提升自身指标鲁棒性，或寻找符合需求的鲁棒指标。最新结果可在线查阅：https://videoprocessing.ai/benchmarks/metrics-robustness.html