The lottery ticket hypothesis (LTH) has attracted attention because it can explain why over-parameterized models often show high generalization ability. It is known that when we use iterative magnitude pruning (IMP), which is an algorithm to find sparse networks with high generalization ability that can be trained from the initial weights independently, called winning tickets, the initial large learning rate does not work well in deep neural networks such as ResNet. However, since the initial large learning rate generally helps the optimizer to converge to flatter minima, we hypothesize that the winning tickets have relatively sharp minima, which is considered a disadvantage in terms of generalization ability. In this paper, we confirm this hypothesis and show that the PAC-Bayesian theory can provide an explicit understanding of the relationship between LTH and generalization behavior. On the basis of our experimental findings that flatness is useful for improving accuracy and robustness to label noise and that the distance from the initial weights is deeply involved in winning tickets, we offer the PAC-Bayes bound using a spike-and-slab distribution to analyze winning tickets. Finally, we revisit existing algorithms for finding winning tickets from a PAC-Bayesian perspective and provide new insights into these methods.

翻译：彩票假设(LTH)吸引了人们的注意,因为它可以解释为什么过度参数化的模型往往表现出高度的概括化能力。众所周知,当我们使用迭代规模的剪切(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP))(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(IMP)(I)(IMP)(IMP)(IMP)(I)(IMP(IMP)(IMP)(I(IMP)(I)(IMP)(IMP(IMP)(IMP(IMP(I)(IMP)(I)(IMP(IMP)(IMP(IMP)(IMP(IMP))(I)(IMP)(I)(I)(IMP(IMP(IMP)(IMP)(IMP))(I)(IMP(IMP)(I)(IMP)(I)(IMP(I)(I)(I)(IMP)))(I))(I)(IMP)(I)(I)(I)(I(I)(I)(Ig)(Ig)(Ig)(Ig)(Ig)(Ig)(Ig)(Ig)(I(Ig)(Ig)(I)(I(I)(I(I(I(I(I)))))(I)(I)(I(I)(I)(I(I(I)(I)(I)(I)(I)(I)(I))(I)(I)(IMP)(I)))(I)(I)(I)(I)(I)(IMP)(I))(I)(IMP)(I)(I(I)(I(I(IMP)(IMP)(IMP