Randomly initialized dense networks contain subnetworks that achieve high accuracy without weight learning -- strong lottery tickets (SLTs). Recently, Gadhikar et al. (2023) demonstrated that SLTs can also be found within a randomly pruned source network, thus reducing the SLT search space. However, this limits the search to SLTs that are even sparser than the source, leading to worse accuracy due to unintentionally high sparsity. This paper proposes a method that reduces the SLT search space by an arbitrary ratio independent of the desired SLT sparsity. A random subset of the initial weights is excluded from the search space by freezing it -- i.e., by either permanently pruning them or locking them as a fixed part of the SLT. In addition to reducing search space, the proposed random freezing can also provide the benefit of reducing the model size for inference. Furthermore, experimental results show that the proposed method finds SLTs with better accuracy-to-model size trade-off than the SLTs obtained from dense or randomly pruned source networks. In particular, the SLTs found in Frozen ResNets on image classification using ImageNet significantly improve the accuracy-to-search space and accuracy-to-model size trade-offs over SLTs within dense (non-freezing) or sparse (non-locking) random networks.

翻译：随机初始化的密集网络包含无需权重学习即可实现高精度的子网络——强彩票券（SLTs）。最近，Gadhikar等人（2023）证明，SLTs也可以在随机剪枝的源网络中找到，从而缩小了SLT的搜索空间。然而，这限制了搜索范围，仅能发现比源网络更稀疏的SLTs，由于无意中产生的高稀疏性，导致精度下降。本文提出一种方法，能够以独立于目标SLT稀疏度的任意比例缩小SLT搜索空间。通过冻结初始权重的一个随机子集——即永久剪除这些权重或将其锁定为SLT的固定部分——将其排除在搜索空间之外。除了缩小搜索空间，所提出的随机冻结方法还能带来减少推理时模型规模的好处。此外，实验结果表明，与从密集或随机剪枝的源网络中获得的SLTs相比，所提方法找到的SLTs在精度与模型规模的权衡上表现更优。具体而言，在使用ImageNet进行图像分类的冻结ResNet中发现的SLTs，相较于在密集（非冻结）或稀疏（非锁定）随机网络中找到的SLTs，在精度与搜索空间以及精度与模型规模的权衡上均有显著提升。