Research in machine unlearning (MU) has gained strong momentum: MU is now widely regarded as a critical capability for building safe and fair AI. In parallel, research into transformer architectures for computer vision tasks has been highly successful: Increasingly, Vision Transformers (VTs) emerge as strong alternatives to CNNs. Yet, MU research for vision tasks has largely centered on CNNs, not VTs. While benchmarking MU efforts have addressed LLMs, diffusion models, and CNNs, none exist for VTs. This work is the first to attempt this, benchmarking MU algorithm performance in different VT families (ViT and Swin-T) and at different capacities. The work employs (i) different datasets, selected to assess the impacts of dataset scale and complexity; (ii) different MU algorithms, selected to represent fundamentally different approaches for MU; and (iii) both single-shot and continual unlearning protocols. Additionally, it focuses on benchmarking MU algorithms that leverage training data memorization, since leveraging memorization has been recently discovered to significantly improve the performance of previously SOTA algorithms. En route, the work characterizes how VTs memorize training data relative to CNNs, and assesses the impact of different memorization proxies on performance. The benchmark uses unified evaluation metrics that capture two complementary notions of forget quality along with accuracy on unseen (test) data and on retained data. Overall, this work offers a benchmarking basis, enabling reproducible, fair, and comprehensive comparisons of existing (and future) MU algorithms on VTs. And, for the first time, it sheds light on how well existing algorithms work in VT settings, establishing a promising reference performance baseline.

翻译：机器学习遗忘（MU）研究已获得强劲发展势头：MU目前被广泛视为构建安全公平人工智能的关键能力。与此同时，计算机视觉任务的Transformer架构研究取得了显著成功：视觉Transformer（VTs）正日益成为CNN的强大替代方案。然而，视觉任务的MU研究主要集中于CNN而非VTs。尽管现有基准测试已涵盖LLM、扩散模型和CNN，但尚未有针对VTs的系统性评估。本研究首次尝试填补这一空白，在不同VT架构（ViT与Swin-T）及不同模型容量下对MU算法性能进行基准测试。研究采用（i）多种数据集以评估数据规模与复杂度的影响；（ii）代表不同MU方法论的多种算法；（iii）单次遗忘与持续遗忘两种协议。此外，本研究重点关注利用训练数据记忆化的MU算法基准测试，因为近期研究发现利用记忆化能显著提升原有SOTA算法的性能。研究过程中，我们系统分析了VTs相对于CNN的训练数据记忆化特性，并评估了不同记忆化代理指标对性能的影响。基准测试采用统一评估指标，涵盖两种互补的遗忘质量度量，以及未见过（测试）数据与保留数据的准确率。总体而言，本研究为VTs上现有（及未来）MU算法提供了可复现、公平且全面的比较基准，首次揭示了现有算法在VT环境中的实际效能，并建立了具有参考价值的性能基线。